git.s-ol.nu alive / 8f6738f
docs: update extension guide s-ol 5 months ago
4 changed file(s) with 282 addition(s) and 226 deletion(s). Raw diff Collapse all Expand all
1212 style = nil
1313 template = 'docs'
1414 dir = 'docs/internals'
15 topics = {'docs/internals/extensions.md'}
15 use_markdown_titles = true
16 topics = {'docs/internals/writing_extensions.md'}
00 You can use this directory to add extensions to alv.
11 See the extension documentation here for more information:
22
3 https://alv.s-ol.nu/stable/internals/topics/extensions.md.html
3 https://alv.s-ol.nu/stable/internals/topics/writing_extensions.md.html
+0
-224
docs/internals/extensions.md less more
0 # writing `alv` extensions
1
2 Extensions for `alv` are implemented in [Lua][lua] or [MoonScript][moonscript]
3 (which runs as Lua). When an `alv` module is [`(require)`][builtins-req]d,
4 alv looks for a Lua module `alv-lib.[module]`. You can simply add a new file
5 with extension `.lua` or `.moon` in the `alv-lib` directory of your alv
6 installation or somewhere else in your `LUA_PATH`.
7
8 To write extensions, a number of classes and utilities are required. All of
9 these are exported in the `base` module.
10
11 ## documentation metadata
12 The lua module should return a `Scope` or a table that will be converted using
13 `Scope.from_table`. All exports should be documented using `Constant.meta`,
14 which attaches a `meta` table to the value that is used for error messages,
15 documentation generation and [`(doc)`][builtins-doc].
16
17 import Constant from require 'alv.base'
18
19 two = Constant.meta
20 meta:
21 name: 'two'
22 summary: "the number two"
23 value: 2
24
25 {
26 :two
27 }
28
29 In the `meta` table `summary` is the only required key, but all of the
30 information that applies should be provided.
31
32 - `name`: the name of this export (for error reporting).
33 - `summary`: a one-line plain-text description of this entry. Should be
34 capitalized and end with a period.
35 - `examples`: a table of strings, each of which is a short one-line code
36 example illustrating the argument names for an Op.
37 - `description`: a longer markdown-formatted description of the functionality
38 of this entry.
39
40 ## defining `Op`s
41 Most extensions will want to define a number of *Op*s to be used by the user.
42 They are implemented by deriving from the `Op` class and implementing at least
43 the `Op:setup` and `Op:tick` methods.
44
45 import Constant, SigStream, Op, Input, evt from require 'alv.base'
46
47 total_sum = Constant.meta
48 meta:
49 name: 'total-sum'
50 summary: "Keep a total of incoming numbers."
51 examples: { '(total-sum num!)' }
52 description: "Keep a total sum of incoming number events, extension-style."
53
54 value: class extends Op
55 new: (...) =>
56 super ...
57 @state or= { total: 0 }
58 @out or= SigStream 'num', @state.total
59
60 setup: (inputs, scope) =>
61 num = evt.num\match inputs
62 super num: Inputs.hot num
63
64 tick: =>
65 @state.total += @inputs.num!
66 @out\set @state.total
67
68 {
69 'total-sum': total_sum
70 }
71
72 ### Op:setup
73 `Op:setup` is called once every *eval cycle* to parse the Op's arguments, check
74 their types, choose the updating behaviour and define the output type.
75
76 The arguments to `:setup` are a list of inputs (each is a `Result` instance),
77 and the `Scope` the evaluation happened in. Ops generally shouldn't use the
78 scope, but might look up 'magic' dynamic symbols like `\*clock\*`.
79
80 #### argument parsing
81 Arguments should be parsed using `base.match`. The two exports `base.match.sig`
82 and `base.match.evt` are used to build complex patterns that can parse and
83 validate the Op arguments into complex structures (see the module documentation
84 for more information).
85
86 import sig, evt from require 'alv.base'
87
88 pattern = evt.bang + sig.str + sig.num*3 + -evt!
89 { trig, str, numbers, optional } = pattern\match inputs
90
91 This example matches first an `EvtStream` of type `bang`, then a `SigStream`
92 of type `str`, followed by one, two or three `num`-values and finally an
93 optional argument `EvtStream` of any type. `:match` will throw an error if it
94 couldn't (fully) match the arguments and otherwise return a structured mapping
95 of the inputs.
96
97 If there are more complex dependencies between arguments, it is recommended to
98 do as much of the parsing as possible using the `base.match` and then continue
99 manually. For invalid or missing arguments, `Error` instances should be thrown
100 using `error` or `assert`.
101
102 #### input setup
103 There are two types of inputs: `Input.hot` and `Input.cold`:
104
105 *Cold* inputs do not cause the Op to update when changes to the input stream
106 are made. They are useful to 'ignore' changes to inputs which are only relevant
107 when another input changed value. Imagine for example a `send-value-when` Op,
108 which sends a value only when a `bang!` input is live. This Op doesn't have to
109 update when the value changes, it's enough to update only when the trigger
110 input changes and simply read the value in that moment.
111
112 *Hot* inputs on the other hand mark the input stream as a dependency for the
113 Op. Depending on the type of `Result`, the semantics are a little different:
114
115 - For `SigStream`s, the Op updates whenever the current value changes. When
116 an input stream is swapped out for another one at evaltime, but their values
117 are momentarily equal, the input is not considered dirty.
118 - For `EvtStream`s and `IOStream`s, the Op updates whenever the stream is
119 dirty. There is no special handling when the stream is swapped out at
120 evaltime.
121
122 All `Result`s from the `inputs` argument that are taken into consideration
123 should be wrapped in an `Input` instance using either `Input.hot` or
124 `Input.cold`, and need to be passed to the `Op:setup` super implementation.
125 To illustrate with the `send-value-when` example:
126
127 setup: (inputs, scope) =>
128 { trig, value } = match 'bang! any', inputs
129
130 super
131 trig: Inputs.hot trig
132 value: Inputs.cold value
133
134 `Op:setup` takes a table that can have any (even nested) shape you want, as
135 long as all 'leaf values' are `Input` instances. The following are both valid:
136
137 super { (Inputs.hot trig), (Inputs.cold value) }
138
139 super
140 trigger: Inputs.hot trig
141 values: { (Inputs.cold a), (Inputs.cold b), (Inputs.cold c) }
142
143 #### output setup
144 When `Op:setup` finishes, `@out` has to be set to a `Result` instance. The
145 instance can be created in `Op:setup`, or by overriding the constructor and
146 delegating to the original one using `super`. In general setting it in the
147 constructor is preferred, and it is only moved to `Op:setup` if the output
148 type depends on the arguments received.
149
150 There are four types of `Result`s that can be created:
151
152 - `SigStream`s track *continuous values*. They can only have one value per
153 tick, and downstream Ops will not update when a *SigStream* has been set
154 to the same value it already had. They are updated using `SigStream:set`.
155 - `EvtStream`s transmit *momentary events*. They can transmit multiple events
156 in a single tick. `EvtStream`s do not keep a value set on the last tick on
157 the next tick. They are updated using `EvtStream:set`.
158 - `IOStream`s are like `EvtStream`s, but their `IOStream:poll` method is
159 polled by the event loop at the start of every tick. This gives them a chance
160 to effectively create changes 'out of thin air' and kickstart the execution
161 of the dataflow engine. All *runtime* execution is due to an `IOStream`
162 becoming dirty somewhere. See the section on implementing `IOStream`s below
163 for more information.
164 - `Constant`s do not change in-between evalcycles. Usually Ops do not output
165 `Constant`s directly, althrough `SigStream`s outputs are automatically
166 'downgraded' to `Constant`s when the Op has no reactive inputs.
167
168 ### Op:tick
169 `Op:tick` is called whenever any of the inputs are *dirty*. This is where the
170 Op's main logic will go. Generally here it should be checked which input(s)
171 changed, and then internal state and the output value may be updated.
172
173 ## defining `Builtin`s
174 Builtins are more powerful than Ops, because they control whether, which and
175 how their arguments are evaluated. They roughly correspond to *macros* in Lisps.
176 There is less of a concrete guideline for implementing Builtins because there
177 are a lot more options, and it really depends a lot on what the Builtin should
178 achieve. Nevertheless, a good starting point is to read the `Builtin` class
179 documentation, take a look at `Builtin`s in `alv/builtins.moon` and get
180 familiar with the relevant internal interfaces (especially `AST`, `Result`, and
181 `Scope`).
182
183 ## defining `IOStream`s
184 `IOStream`s are `EvtStream`s that can 'magically' create events out of
185 nothing. They are the source of all processing in alv. Whenever you want to
186 bring events into alv from an external protocol or application, an IOStream
187 will be necessary.
188
189 To implement a custom IOStream, create it as a class that inherits from the
190 `IOStream` base and implement the constructor and `IOStream:poll`:
191
192 import T, IOStream from require 'alv.base'
193
194 class UnreliableStream extends IOStream
195 new: => super T.bang
196
197 poll: =>
198 if math.random! < 0.1
199 @set true
200
201 In the constructor, you should call the super-constructor `EvtStream.new` to
202 set the event type. Often this will be a custom event that is only used inside
203 your extension (such as e.g. the `midi/port` type in the [midi][modules-midi]
204 module), but it can also be a primitive type like `T.bang` in this example. In
205 `:poll`, your IOStream is given a chance to communicate with the external world
206 and create any resulting events. The example stream above randomly sends bang
207 events out, with a 10% chance each 'tick' of the system. Note that there is no
208 guarantee about when or how often ticks occur, so you really shouldn't rely on
209 them this way in a real extension.
210
211 ### using `IOStream`s
212 There's a couple of ways IOStreams can be used and exposed to the user of your
213 extension. You can either expose an instance of your IOStream directly
214 (documented using `SigStream.meta`), or offer an Op that creates and returns
215 an instance in `Op.out` - that way the IOStream can be created only on demand
216 and take parameters. It is also possible to not exepose the IOStream at all,
217 and rather pass it as a hardcoded input into an Op's `Op.inputs`.
218
219 [lua]: https://www.lua.org/
220 [moonscript]: http://moonscript.org/
221 [builtins-req]: ../../reference/index.html#require
222 [builtins-doc]: ../../reference/index.html#doc
223 [modules-midi]: ../../reference/midi.html
0 # writing `alv` extensions
1
2 Extensions for `alv` are implemented in [Lua][lua] or [MoonScript][moonscript]
3 (which runs as Lua). When an `alv` module is [`(require)`][builtins-req]d,
4 alv looks for a Lua module `alv-lib.[module]`. You can simply add a new file
5 with extension `.lua` or `.moon` in the `alv-lib` directory of your alv
6 installation or somewhere else in your `LUA_PATH`.
7
8 To write extensions, a number of classes and utilities are required. All of
9 these are exported in the `base` module.
10
11 ## alv values
12 In the alv runtime, values are represented as instances of one of the three
13 classes implementing the `Result` interface; `Constant`, `SigStream` or
14 `EvtStream`.
15
16 A `Result` contains a type, the "unwrapped" Lua value, and optional metadata.
17
18 ### types
19 Different types are represented as instances of the `type.Type` interface.
20 Such types can be @{type.Primitive|Primitive} types (which are opaque to alv
21 user code), @{type.Array|Array}s or @{type.Struct|Struct}s.
22
23 @{type.Primitive|Primitive} types are identified simply as a string.
24 A primitive type should have a well-defined Lua equivalent that implementations
25 can expect when unwrapping a corresponding alv value. Here is how the types
26 used by alv and the standard library map to Lua values:
27
28 - `num`: Lua `number`
29 - `str`: Lua `string`
30 - `sym`: Lua `string`
31 - `bool`: Lua `boolean`
32 - `bang`: always Lua `true`
33 - `scope`: `Scope` instance
34 - `fndef`: `FnDef` instance
35 - `opdef`: class inheriting from `Op` or `PureOp`
36 - `builtin`: class inheriting from `Builtin`
37
38 New primitive types can be created by extensions to represent values that should be
39 opaque to other extensions and alv code. To avoid namespace collisions, such
40 primitive types should be prefixed with the extension name and a slash.
41 For example, the `love` extension uses the type `love/shape` internally.
42
43 To obtain primitive type instances easily, the `type.T` "magic table" is
44 provided. Simply indexing in this table will produce a cached
45 @{type.Primitive|Primitive} instance:
46
47 import T from require 'alv.base'
48
49 number_type = T.num
50 shape_type = T['love/shape']
51
52 @{type.Array|Array}s and @{type.Struct|Struct}s are composite types that
53 contain other types.
54
55 Arrays contain a fixed number of elements of a single type. For example,
56 this code defines a "vec3" type that consists of three numbers:
57
58 import T, Array from require 'alv.base'
59 vec3 = Array 3, T.num
60
61 Structs contain a set of labelled values that can each have a different type.
62 This code snippet defines a "person" type with two keys, "name" and "age".
63
64 import T, Struct from require 'alv.base'
65 person = Struct { name: T.str, age: T.num }
66
67 `Type` instances provide shorthand methods to create instances of the three
68 *kinds* of `Result`:
69
70 word = T.str\mk_const "hello" -- value required
71 odd_number = T.num\mk_sig 7 -- initial value (can be provided later)
72 emails = T["email/message"]\mk_evt!
73
74 ### metadata and documentation
75 Using `Constant.meta`, documentation metadata can also be attached to values.
76 This metadata is used for error messages, documentation generation and the
77 [`(doc)`][builtins-doc] builtin.
78
79 In the `meta` table `summary` is the only required key, but all of the
80 information that applies should be provided.
81
82 - `name`: the name of this export (for error reporting).
83 - `summary`: a one-line plain-text description of this entry. Should be
84 capitalized and end with a period.
85 - `examples`: a table of strings, each of which is a short one-line code
86 example illustrating the argument names for an Op.
87 - `description`: a longer markdown-formatted description of the functionality
88 of this entry.
89
90 ## module format
91 The lua module should return a `Result` which will be returned as the result
92 from [`(require)`][builtins-require]. In almost all cases, the return value
93 should be a `Scope` containing individual `Result`s that can be imported
94 together using [`(import)`][builtins-imp] and [`(import*)`][builtins-im_].
95
96 `Constant.meta` calls `Constant.wrap`, which will automatically turn raw tables
97 into `Scope`s and label other Lua primitive types correctly.
98
99 import Constant from require 'alv.base'
100
101 -- define some values
102 one = Constant.meta
103 meta:
104 name: 'one'
105 summary: "the number one"
106 value: 1
107
108 two = Constant.meta
109 meta:
110 name: 'two'
111 summary: "the number two"
112 value: 2
113
114 -- define and return a Constant of type "scope"
115 -- that contains our exports
116 Constant.meta
117 meta:
118 name: 'numbers'
119 summary: "a module containing common numbers."
120 value: { :one, :two }
121
122 ## defining `Op`s
123 Most extensions will want to define a number of *Op*s to be used by the user.
124 They are implemented by deriving from the `Op` class and implementing at least
125 the `Op:setup` and `Op:tick` methods.
126
127 import Constant, SigStream, Op, Input, evt from require 'alv.base'
128
129 total_sum = Constant.meta
130 meta:
131 name: 'total-sum'
132 summary: "Keep a total of incoming numbers."
133 examples: { '(total-sum num!)' }
134 description: "Keep a total sum of incoming number events, extension-style."
135
136 value: class extends Op
137 new: (...) =>
138 super ...
139 @state or= { total: 0 }
140 @out or= SigStream 'num', @state.total
141
142 setup: (inputs, scope) =>
143 num = evt.num\match inputs
144 super num: Inputs.hot num
145
146 tick: =>
147 @state.total += @inputs.num!
148 @out\set @state.total
149
150 Constant.meta
151 meta:
152 name: 'my-module'
153 description: "This is my own awesome module."
154 value: { 'total-sum': total_sum }
155
156 ### Op:setup
157 `Op:setup` is called once every *eval cycle* to parse the Op's arguments, check
158 their types, choose the updating behaviour and define the output type.
159
160 The arguments to `:setup` are a list of inputs (each is a `Result` instance),
161 and the `Scope` the evaluation happened in. Ops generally shouldn't use the
162 scope, but might look up 'magic' dynamic symbols like `\*clock\*`.
163
164 #### argument parsing
165 Arguments should be parsed using `base.match`. `base.match.const`, `base.match.sig`
166 and `base.match.evt` are used to build complex patterns that can parse and
167 validate the Op arguments into complex structures (see the module documentation
168 for more information).
169
170 import sig, evt from require 'alv.base'
171
172 pattern = evt.bang + sig.str + sig.num*3 + -evt!
173 { trig, str, numbers, optional } = pattern\match inputs
174
175 This example matches first an `EvtStream` of type `bang`, then a `SigStream`
176 of type `str`, followed by one, two or three `num`-values, and finally an
177 optional argument `EvtStream` of any type. `:match` will throw an error if it
178 couldn't (fully) match the arguments and otherwise return a structured mapping
179 of the inputs.
180
181 If there are more complex dependencies between arguments, it is recommended to
182 do as much of the parsing as possible using the `base.match` and then continue
183 manually. For invalid or missing arguments, `Error` instances should be thrown
184 using `error` or `assert`.
185
186 #### input setup
187 There are two types of inputs: `Input.hot` and `Input.cold`:
188
189 *Cold* inputs do not cause the Op to update when changes to the input stream
190 are made. They are useful to 'ignore' changes to inputs which are only relevant
191 when another input changed value. Imagine for example a `send-value-when` Op,
192 which sends a value only when a `bang!` input is live. This Op doesn't have to
193 update when the value changes, it's enough to update only when the trigger
194 input changes and simply read the value in that moment.
195
196 *Hot* inputs on the other hand mark the input stream as a dependency for the
197 Op. Depending on the type of `Result`, the semantics are a little different:
198
199 - For `SigStream`s, the Op updates whenever the current value changes. When
200 an input stream is swapped out for another one at evaltime, but their values
201 are momentarily equal, the input is not considered dirty.
202 - For `EvtStream`s and `IOStream`s, the Op updates whenever the stream is
203 dirty. There is no special handling when the stream is swapped out at
204 evaltime.
205
206 All `Result`s from the `inputs` argument that are taken into consideration
207 should be wrapped in an `Input` instance using either `Input.hot` or
208 `Input.cold`, and need to be passed to the `Op:setup` super implementation.
209 To illustrate with the `send-value-when` example:
210
211 pattern = evt.bang + sig!
212 setup: (inputs, scope) =>
213 { trig, value } = pattern\match inputs
214
215 super
216 trig: Inputs.hot trig
217 value: Inputs.cold value
218
219 `Op:setup` takes a table that can have any (even nested) shape you want, as
220 long as all 'leaf values' are `Input` instances. The following are both valid:
221
222 super { (Inputs.hot trig), (Inputs.cold value) }
223
224 super
225 trigger: Inputs.hot trig
226 values: { (Inputs.cold a), (Inputs.cold b), (Inputs.cold c) }
227
228 #### output setup
229 When `Op:setup` finishes, `@out` has to be set to a `Result` instance. The
230 instance can be created in `Op:setup`, or by overriding the constructor and
231 delegating to the original one using `super`. In general setting it in the
232 constructor is preferred, and it is only moved to `Op:setup` if the output
233 type depends on the arguments received.
234
235 There are three types of `Result`s that can be created:
236
237 - `SigStream`s track *continuous values*. They can only have one value per
238 tick, and downstream Ops will not update when a *SigStream* has been set
239 to the same value it already had. They are updated using `SigStream:set`.
240 - `EvtStream`s transmit *momentary events*. They can transmit multiple events
241 in a single tick. `EvtStream`s do not keep a value set on the last tick on
242 the next tick. They are updated using `EvtStream:set`.
243 - `Constant`s do not change in-between evalcycles. Usually Ops do not output
244 `Constant`s directly, as `SigStream`s outputs are automatically
245 'downgraded' to `Constant`s when the Op has no reactive inputs.
246
247 ### Op:tick
248 `Op:tick` is called whenever any of the inputs are *dirty*. This is where the
249 Op's main logic will go. Generally here it should be checked which input(s)
250 changed, and then internal state and the output value may be updated.
251
252 - @TODO: explain `Op:tick` setup argument
253 - @TODO: explain how to use `Input:dirty`
254 - @TODO: explain `Op:unwrap_all`
255
256 ### state and forking
257 - @TODO: explain `Op:fork`
258
259 ### IO ops and polling
260 - @TODO: explain `Op:poll` and "IO Ops"
261
262 ## defining `Builtin`s
263 Builtins are more powerful than Ops because they control whether, how and
264 when their arguments are evaluated. They roughly correspond to *macros* in Lisps.
265 There is less of a concrete guideline for implementing Builtins because there
266 are a lot more options, and it really depends a lot on what the Builtin should
267 achieve. Nevertheless, a good starting point is to read the `Builtin` class
268 documentation, take a look at `Builtin`s in `alv/builtins.moon` and get
269 familiar with the relevant internal interfaces (especially `AST`, `Result`, and
270 `Scope`).
271
272 [lua]: https://www.lua.org/
273 [moonscript]: http://moonscript.org/
274 [builtins-req]: ../../reference/index.html#require
275 [builtins-imp]: ../../reference/index.html#import
276 [builtins-im_]: ../../reference/index.html#import*
277 [builtins-doc]: ../../reference/index.html#doc
278 [modules-midi]: ../../reference/midi.html