Compare commits
No commits in common. "a88229620212c451092a670d155a70d46c564f18" and "3d68ef1e974b6cba15317d96f5ccc37f14697ea6" have entirely different histories.
a882296202
...
3d68ef1e97
1
PLUGS.md
1
PLUGS.md
@ -3,4 +3,5 @@ This file lists all plugs that SilverBullet will load. Run the {[Plugs: Update]}
|
||||
```yaml
|
||||
- github:silverbulletmd/silverbullet-git/git.plug.js
|
||||
- github:silverbulletmd/silverbullet-mermaid/mermaid.plug.js
|
||||
- github:joekrill/silverbullet-treeview/treeview.plug.js
|
||||
```
|
||||
@ -1,60 +0,0 @@
|
||||
# NODE_REGISTRY
|
||||
|
||||
# How to store nodes in the registry?
|
||||
|
||||
The nodes need:
|
||||
|
||||
## ID: string/string/string
|
||||
I could imagine something like jimfx/nature/branch
|
||||
|
||||
## INPUTS:
|
||||
We need a way to define how the UI should render the input ui. Could be something like:
|
||||
|
||||
```json
|
||||
{
|
||||
"inputs": [
|
||||
“a”: {
|
||||
“type”: “select”,
|
||||
“options”: [
|
||||
“curly”,
|
||||
“straight”
|
||||
]
|
||||
},
|
||||
"b": {
|
||||
"type":"float",
|
||||
"value": 0.5,
|
||||
"max": 10.0,
|
||||
"min": 0
|
||||
},
|
||||
"res":{
|
||||
"type": "integer",
|
||||
"value": 5,
|
||||
"setting":"leaves/resolution"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## OUTPUTS
|
||||
The nodes output, could look something like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"outputs": ["stem","float"]
|
||||
}
|
||||
```
|
||||
|
||||
# How to store serialized nodes
|
||||
|
||||
```json
|
||||
{
|
||||
"id":"jimfx/nature/branch",
|
||||
"inputs": [0,8.4],
|
||||
"position": [0.6, 10]
|
||||
"meta": {
|
||||
"title":"Branch Node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 81 KiB |
@ -1,20 +1,76 @@
|
||||
|
||||
```toc
|
||||
```
|
||||
|
||||
# Notes
|
||||
|
||||
We need three major components to work together. I would love it if the node-store and the runtime are super loosely coupled so that they could be replaced by at will. So for example i could use the “server-runtime” that executes the node-graph on the server, or the “local” runtime that executes the node-graph locally.
|
||||
|
||||
## [[Projects/bachelor/NODE_INTERFACE]]
|
||||
|
||||
**NODE_INTERFACE**
|
||||
This is where the user interacts with the node graph. The frontend loads a node-system. Then fetches all the relevant nodes from the node-store.
|
||||
|
||||
## [[Projects/bachelor/NODE_REGISTRY]]
|
||||
**NODE_REGISTRY**
|
||||
The node-store stores all the nodes. For each node it stores a definition and the wasm blob that executes that node.
|
||||
|
||||
## [[Projects/bachelor/RUNTIME_EXECUTOR]]
|
||||
**[[Projects/bachelor/RUNTIME_EXECUTOR|RUNTIME_EXECUTOR]]**
|
||||
The runtime gets a node-graph and returns the result after executing the node-graph. It fetches the relevant nodes from the node-store.
|
||||
|
||||
|
||||
This is how these three components would communicate
|
||||
|
||||
# How to store nodes in the registry?
|
||||
|
||||
The nodes need:
|
||||
|
||||
## ID: string/string/string
|
||||
I could imagine something like jimfx/nature/branch
|
||||
|
||||
## INPUTS:
|
||||
We need a way to define how the UI should render the input ui. Could be something like:
|
||||
|
||||
```json
|
||||
{
|
||||
"inputs": [
|
||||
“a”: {
|
||||
“type”: “select”,
|
||||
“options”: [
|
||||
“curly”,
|
||||
“straight”
|
||||
]
|
||||
},
|
||||
"b": {
|
||||
"type":"float",
|
||||
"value": 0.5,
|
||||
"max": 10.0,
|
||||
"min": 0
|
||||
},
|
||||
"res":{
|
||||
"type": "integer",
|
||||
"value": 5,
|
||||
"setting":"leaves/resolution"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## OUTPUTS
|
||||
The nodes output, could look something like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"outputs": ["stem","float"]
|
||||
}
|
||||
```
|
||||
|
||||
# How to store serialized nodes
|
||||
|
||||
```json
|
||||
{
|
||||
"id":"jimfx/nature/branch",
|
||||
"inputs": [0,8.4],
|
||||
"position": [0.6, 10]
|
||||
"meta": {
|
||||
"title":"Branch Node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.4 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB |
@ -1,6 +1,4 @@
|
||||
# RUNTIME_EXECUTOR
|
||||
```toc
|
||||
```
|
||||
|
||||
This component receives a serialized node_graph and returns the output. I generally love the idea that a node is a pure function which just receives its own inputs and then calculates them. BUT! there is always a but, there are some things that the nodes needs that are not its own inputs. These are in my experience:
|
||||
|
||||
@ -15,6 +13,7 @@ After some thinking, my idea is to model these inputs as *hidden* inputs to the
|
||||
But they are gonna be passed to the nodes as normal inputs. This could be super useful to allow easy memoization of the nodes, because if none of the inputs changed, we can return the previously computed output.
|
||||
|
||||
|
||||
|
||||
## Linking/Importing of Nodes
|
||||
|
||||
**WebAssembly Components**
|
||||
@ -25,208 +24,3 @@ For this the runtime_executor needs to be wasm aswell.
|
||||
**No linking**
|
||||
\+ Very easy to implement, works everywhere
|
||||
\- Probably not as fast as components
|
||||
|
||||
|
||||
# Handle multiple outputs
|
||||
|
||||
The general idea is to model the nodes as a visual representation of pure functions, that means the function only operates on it’s own inputs, doesn’t have any side effects and always produces the same output for the same input.
|
||||
|
||||
For very simple graph, such as the following this works very well.
|
||||
|
||||
The `math/add` node could be modelled very simple as the following pure function:
|
||||
|
||||
```rust
|
||||
fn math_add(a: i32, b:i32) -> i32 {
|
||||
return a + b
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
The problem occurs if we try do adher to this principle for more complex use cases.
|
||||
|
||||
## Problem Example 1
|
||||
|
||||
|
||||
Users would probably expect the random node to output for different results. If we would model the `random` node as a pure function, it does not know how many results it should generate.
|
||||
|
||||
## Problem Example 2
|
||||
|
||||
On the left you see a node setup for the procedural generation of geometry. On the right one of the probably outcomes a user would expect.
|
||||
|
||||
If we model the noise function as a pure function we would first have to figure out how to pass the result of the noise function as a pure value to the `cylinder` node. The `noise` node also has no idea how many outputs are connected to it.
|
||||
|
||||
|
||||
## Solution 1 (Power to the Nodes)
|
||||
|
||||
We could solve this by letting each node execute its parent node graph however it wants. This would allow the `cylinder` node to execute the `noise` however often it needs to create the cylinders.
|
||||
|
||||
|
||||
```typescript
|
||||
// how this could look like in the node
|
||||
|
||||
{
|
||||
id: "cylinder",
|
||||
inputs: {
|
||||
"amount": { type: "integer", value: 1 },
|
||||
"height": { type: "float" },
|
||||
"radius": { type: "float" },
|
||||
},
|
||||
execute: ({ amount, height, width }, executeNodeGraph) => {
|
||||
|
||||
const stemAmount = executeNodeGraph(amount);
|
||||
const ringAmount = 10;
|
||||
|
||||
for(let i = 0; i < stemAmount; i++){
|
||||
for(let x = 0; x < ringAmount; x++){
|
||||
const radius = executeNodeGraph(width);
|
||||
// create a single ring
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**
|
||||
- nodes become very flexible on how the work in their inputs
|
||||
|
||||
**Cons**
|
||||
- Every Nodes needs access to the [[Projects/bachelor/RUNTIME_EXECUTOR]]
|
||||
- It could become very unpredictable how each node behaves, because the each have a own way to handle inputs
|
||||
|
||||
## Solution 2 (Parametrized nodes, embed execution)
|
||||
|
||||
We could change the output of the noise function to output a definition of noise like the following:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "noise",
|
||||
"size": 0.7,
|
||||
"strength": 1.5
|
||||
}
|
||||
```
|
||||
|
||||
Or the random node could output something like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "random",
|
||||
"min": 0,
|
||||
"max": 42
|
||||
}
|
||||
```
|
||||
|
||||
Then we could embed a function that nows how to turn these parametrized nodes into numbers inside the `cylinder` node. One problem with this is that this function needs to be recursive, because the inputs of the parametrized nodes could also be other parametrized nodes:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "noise",
|
||||
"size": 0.7,
|
||||
"strength": {
|
||||
"type": "random",
|
||||
"min": 0,
|
||||
"max": 42
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**
|
||||
- The overhead inside the `cylinder` node is smaller than the other solutions
|
||||
- BIIIG Pro; the runtime knows nothing about that
|
||||
|
||||
**Cons**
|
||||
- We basically have to embed the execution function of number nodes inside other nodes
|
||||
- recursive execution of the nodes definitely needs memoization
|
||||
|
||||
## Solution 3 (Partially invert the flow)
|
||||
|
||||
Normally the flow of information in this graph is only from the parents to the children nodes. One idea could be for the children nodes to send to their parents what they need, and then have the parents return that result to their children.
|
||||
|
||||
|
||||
## Solution 4 (Add execution information to parameters)
|
||||
|
||||
We could also add metadata to the node definition to tell the [[Projects/bachelor/RUNTIME_EXECUTOR]]. This could either be how it should execute the parent nodes of a node, or directly in the node how it should be executed.
|
||||
|
||||
For example in the first [[#Problem Example 1|Example 1]] it could look like this
|
||||
|
||||
```typescript
|
||||
const randomNode = {
|
||||
id: "random",
|
||||
inputs: {
|
||||
"value": { type: "float", value: 0.1, internal: true },
|
||||
"seed": { type:"runtime_seed" }
|
||||
},
|
||||
outputs: ["float"],
|
||||
execution: {
|
||||
// this is the important part
|
||||
distinctOutput: true
|
||||
},
|
||||
execute: ({ value, seed }) => {
|
||||
return value
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This could tell the [[Projects/bachelor/RUNTIME_EXECUTOR]] to execute the `randomNode` for every of its child nodes individually.
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Parameters Dimensionality
|
||||
|
||||
We could define how many dimensions a parameter can handle and which range it could handle. For example the `radius` parameter could be a 2D parameter, and we could map the x coordinate to the angle relative to the center of the stem and y to the height along the stem.
|
||||
|
||||
The `noise` node would output a 2D parameter. We sample it with 2 coordinates. But we could also only sample it with one coordinate.
|
||||
|
||||
If we would have a theoretical `curve` function, it would produce a `1D` parameter which is sampled with a x coordinate from 0 to 1.
|
||||
|
||||
The `random` node outputs a `1D` parameter, we sample it with the `seed`.
|
||||
|
||||
Then we would need a way to sample this parameter where we need the value.
|
||||
|
||||
```typescript
|
||||
|
||||
const cylinderNode = {
|
||||
id: "cylinder",
|
||||
outputs: ["geometry"],
|
||||
inputs: {
|
||||
height: { type: "float" },
|
||||
radius: { type: "float" },
|
||||
amount: { type: "integer" },
|
||||
seed: { type: "runtime_seed" },
|
||||
xResolution: { type:"integer", setting:"resolution.x" },
|
||||
yResolution: { type:"integer", setting:"resolution.y" },
|
||||
},
|
||||
execute({ height, radius, amount, xResolution, yResolution, seed }){
|
||||
const coordinates: number[][] = [];
|
||||
|
||||
const angleIncrement = (2 * Math.PI) / xResolution;
|
||||
|
||||
const amount = sampleParameter1D(amount);
|
||||
|
||||
for (let j = 0; j < amount; j++){
|
||||
const cylinder: number[] = []
|
||||
|
||||
// here we sample a 1D Parameter
|
||||
const heightIncrement = sampleParameter1D(height, j) / yResolution;
|
||||
|
||||
for (let i = 0; i < yResolution; i++) {
|
||||
const y = i * heightIncrement;
|
||||
|
||||
for (let j = 0; j < xResolution; j++) {
|
||||
const angle = j * angleIncrement;
|
||||
const radius = sampleParameter2D(radius, y, angle);
|
||||
const x = radius * Math.cos(angle);
|
||||
const z = radius * Math.sin(angle);
|
||||
coordinates.push( , y, z);
|
||||
}
|
||||
}
|
||||
coordinates.push(cylinder);
|
||||
}
|
||||
|
||||
return coordinates;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
74
STYLES.md
74
STYLES.md
@ -1,49 +1,45 @@
|
||||
|
||||
|
||||
```space-style
|
||||
|
||||
```css
|
||||
#sb-root {
|
||||
--top-background-color: #1a1a1a;
|
||||
--top-border-color: #222;
|
||||
/* Uncomment the next line to set the editor font to Courier */
|
||||
--editor-font: "Courier" !important;
|
||||
/* Uncomment the next line to set the editor width to 1400px */
|
||||
/* --editor-width: 1400px !important; */
|
||||
}
|
||||
|
||||
/* Choose another header color */
|
||||
#sb-top {
|
||||
box-shadow: 0px 0px 50px rgba(0, 0, 0, 0.1);
|
||||
/* background-color: #ffe54a !important; */
|
||||
}
|
||||
|
||||
#sb-current-page .cm-line {
|
||||
font-size: 0.8em;
|
||||
padding-top: 0.1em !important;
|
||||
/* You can even change the appearance of buttons */
|
||||
button {
|
||||
/* align-items: center;
|
||||
background-color: #fff;
|
||||
border-radius: 6px;
|
||||
box-shadow: transparent 0 0 0 3px,rgba(18, 18, 18, .1) 0 6px 20px;
|
||||
box-sizing: border-box;
|
||||
color: #121212;
|
||||
cursor: pointer;
|
||||
display: inline-flex;
|
||||
flex: 1 1 auto;
|
||||
font-family: Inter,sans-serif;
|
||||
font-size: 0.6rem;
|
||||
justify-content: center;
|
||||
line-height: 1;
|
||||
margin: 0.2rem;
|
||||
outline: none;
|
||||
padding: 0.3rem 0.4rem;
|
||||
text-align: center;
|
||||
text-decoration: none;
|
||||
transition: box-shadow .2s,-webkit-box-shadow .2s;
|
||||
white-space: nowrap;
|
||||
border: 0;
|
||||
user-select: none;
|
||||
-webkit-user-select: none;
|
||||
touch-action: manipulation; */
|
||||
}
|
||||
|
||||
.sb-markdown-widget {
|
||||
padding: 0px !important;
|
||||
border: none !important;
|
||||
button:hover {
|
||||
/* box-shadow: #121212 0 0 0 3px, transparent 0 0 0 0; */
|
||||
}
|
||||
|
||||
/* first line - editing */
|
||||
div:not(.sb-line-fenced-code) + .sb-line-fenced-code {
|
||||
border-radius: 10px 10px 0px 0px;
|
||||
padding-top: 10px !important;
|
||||
padding-right: 10px !important;
|
||||
}
|
||||
|
||||
.sb-line-fenced-code:not(.sb-fenced-code-iframe) {
|
||||
padding-left: 20px !important;
|
||||
padding-right: 20px !important;
|
||||
}
|
||||
|
||||
|
||||
/* last line - editing */
|
||||
.sb-line-fenced-code:has( > .sb-meta) {
|
||||
border-radius: 0px 0px 10px 10px;
|
||||
}
|
||||
|
||||
|
||||
.sb-code-info {
|
||||
padding-right: 2px !important;
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user