Lazy Evaluation
May 31st, 2019 - written by Kimserey with .
Last week we implemented a metacircular evaluator in Racket, evaluating a subset of Racket language. We saw that the main functions part of an evaluator are eval evaluating the meaning of an expression and apply, applying a procedure to arguments. We wrote the evaluator following the order of execution of arguments and procedures from Lisp. Lisp being an applicative-order language, when provided a procedure with parameters, the parameters get evaluated before the procedure is applied. As opposed to normal-order languages, also called lazy languages, which delay the evaluation of arguments until the result of the procedure application is required.
Today we will see the implication of this slight change in order of application by modifying our evaluator created last week, transforming it into a lazy evaluator.
This post is based on our previous post where we built an evaluator for Racket. It is highly recommended to start by reading the previous post.
Normal Order
Lisp being an applicative order language, arguments gets evaluated before the procedure is applied to them. The following example showcases the possibilities offered by a lazy language.
1
(define (do-nothing a b) 'nothing)
We define do-nothing, a procedure which wastefully take a and b and does nothing with it.
1
(do-nothing (/ 1 0) 'b)
Executing this will yield a division by zero error. (/ 1 0) being passed as argument is evaluated in order to apply do-nothing procedure.
The reason why arguments are evaluated first lies in the implementation of the evaluator. When the compound procedure is applied, each operands are evaluated, mapped to their variables, and save in an environment frame (more on last week post).
As opposed to applicative order, normal order languages, also called lazy languages, only evaluate the parameters when needed. Instead of saving the computed value of the arguments, a thunk gets created and saved as value.
Thunks are delayed evaluation of arguments.
Thunk Implementation
In order to support lazy evaluation, we start by changing eval and apply-local. The evaluation of procedure arguments used to occur when an application was found, therefore instead of previously extracting the list of operands, we now then directly pass the operands to the apply.
1
2
3
4
5
6
7
; From eval
[(application? exp)
(apply-local
(eval (operator exp) env)
(list-of-values
(operands exp)
env))]
List-of-values used to evaluate each value and cons‘ing them together.
1
2
3
4
5
6
; From updated lazy eval
[(application? exp)
(apply-local
(actual-value (operator exp) env)
(operands exp)
env)]
Instead of that, we will now simply pass the operands as expressions to apply-local together with the environment. We also introduced actual-value which abstracts away the eval of the operator.
We then change apply-local to take the environment as argument.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
(define (apply-local procedure arguments env)
(cond [(primitive-procedure? procedure)
(apply-primitive-procedure
procedure
(list-of-args-values arguments env))] ; <= For primitive procedures, we evaluate directly the arguments
[(compound-procedure? procedure)
(eval-sequence
(procedure-body procedure)
(extend-environment
(procedure-parameters procedure)
(list-of-delayed-args arguments env) ; <= For compount procedures, we delay arguments
(procedure-environment procedure)))]
[else
(error "Unknown procedure type: APPLY" procedure)]))
And we modified the primitive procedure arguments to use list-of-args-values to evaluate the argument expressions directly and we use list-of-delayed-args to delay the evaluation of argument expressions introducing the laziness.
We also need to modify the eval-if,
1
2
3
4
(define (eval-if exp env)
(if (true? (actual-value (if-predicate exp) env))
(eval (if-consequent exp) env)
(eval (if-alternative exp) env)))
where we have to use actual-value to ensure evaluation of the predicate, whether value or thunk.
We then define actual-value, list-of-args-values and list-of-delayed-args
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(define (actual-value exp env)
(force-it (eval exp env)))
(define (list-of-args-values exps env)
(if (no-operands? exps)
'()
(mcons (actual-value
(first-operand exps)
env)
(list-of-args-values
(rest-operands exps)
env))))
(define (list-of-delayed-args exps env)
(if (no-operands? exps)
'()
(mcons (delay-it
(first-operand exps)
env)
(list-of-delayed-args
(rest-operands exps)
env))))
actual-value calls force-it which forces the execution of the evaluation. List-of-args-values recursively forces the execution of each expression part of the list provided and list-of-delayed-args transforms each expression part of the list provided to a delayed evaluation, a thunk.
Delay-it is implemented by taking the expression and the environment and tagging them with a tag 'thunk.
1
2
(define (delay-it exp env)
(mlist 'thunk exp env))
Then we can create a thunk? predicate looking for the 'thunk tag.
1
2
3
4
5
(define (thunk? obj) (tagged-list? obj 'thunk))
(define (thunk-exp thunk) (mcar (mcdr thunk)))
(define (thunk-env thunk) (mcar (mcdr (mcdr thunk))))
And we also create selectors for the expression and the environment. Lastly we need to define force-it, which simply evaluates any thunk found and once evaluated, replaces the thunk expression and environment by the resulting value.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
(define (force-it obj)
(cond
[(thunk? obj)
(let ([result (actual-value (thunk-exp obj) (thunk-env obj))])
(set-mcar! obj 'evaluated-thunk)
(set-mcar! (mcdr obj) result)
(set-mcdr! (mcdr obj) '())
result)]
[(evaluated-thunk? obj)
(thunk-value obj)]
[else obj]))
(define (evaluated-thunk? obj)
(tagged-list? obj 'evaluated-thunk))
(define (thunk-value evaluated-thunk)
(mcar (mcdr evaluated-thunk)))
If the object provided was already evaluated, we directly return the result of the evaluated thunk. This method of caching result precalculated is called memoization, the concept is explained in my previous post on dynamic programming.
Example
Once we have modified eval and apply, our evaluator will now be completely lazy. Therefore defining the same procedure do-nothing as we did previously,
1
(define (do-nothing a b) 'nothing)
will no longer yield a division by zero exception.
1
(do-nothing (/ 1 0) 'b)
Instead it will delay the parameters until necessary, and since none of the paremeters are used, will return directly 'nothing.
Conclusion
Today we explored the difference between Applicative Order and Normal Order languages. We saw how an Applicative Order language could be turned into a Normal Order language by changing how parameters of application are evaluated and hence providing a functionality built into the language. Normal Order languages are also known as lazy languages where Lisp is a applicative order language. I hope you liked this post and I see you on the next one!
Complete Lazy Evaluator
Compared to our previous implementation of the evaluator, we have to change all list and cons to their mutable equivalent mlist and mcons. The following source code contains the full code implementing the lazy evaluator:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
(require compatibility/mlist)
(define (eval exp env)
(cond [(self-evaluating? exp)
exp]
[(variable? exp)
(lookup-variable-value exp env)]
[(quoted? exp)
(text-of-quotation exp)]
[(assignment? exp)
(eval-assignment exp env)]
[(definition? exp)
(eval-definition exp env)]
[(if? exp)
(eval-if exp env)]
[(lambda? exp)
(make-procedure
(lambda-parameters exp)
(lambda-body exp)
env)]
[(begin? exp)
(eval-sequence
(begin-actions exp)
env)]
[(cond? exp)
(eval (cond->if exp) env)]
[(application? exp)
(apply-local
(actual-value (operator exp) env)
(operands exp) ; Instead of evaluating the operands, we pass them directly to apply-local
env)] ; Apply-local needs the environment as the delayed operands will need to be evaluated when needed
[else
(error "Unknown expression type: EVAL" exp)]))
(define (apply-local procedure arguments env)
(cond [(primitive-procedure? procedure)
(apply-primitive-procedure
procedure
(list-of-args-values arguments env))] ; For primitive procedures, we evaluate directly as applicative-order
[(compound-procedure? procedure)
(eval-sequence
(procedure-body procedure)
(extend-environment
(procedure-parameters procedure)
(list-of-delayed-args arguments env) ; All arguments are delayed
(procedure-environment procedure)))]
[else
(error "Unknown procedure type: APPLY" procedure)]))
(define (actual-value exp env)
(force-it (eval exp env)))
(define (list-of-args-values exps env)
(if (no-operands? exps)
'()
(mcons (actual-value
(first-operand exps)
env)
(list-of-args-values
(rest-operands exps)
env))))
(define (list-of-delayed-args exps env)
(if (no-operands? exps)
'()
(mcons (delay-it
(first-operand exps)
env)
(list-of-delayed-args
(rest-operands exps)
env))))
(define (delay-it exp env)
(mlist 'thunk exp env))
(define (thunk? obj) (tagged-list? obj 'thunk))
(define (thunk-exp thunk) (mcar (mcdr thunk)))
(define (thunk-env thunk) (mcar (mcdr (mcdr thunk))))
(define (evaluated-thunk? obj)
(tagged-list? obj 'evaluated-thunk))
(define (thunk-value evaluated-thunk)
(mcar (mcdr evaluated-thunk)))
(define (force-it obj)
(cond
[(thunk? obj)
(let ([result (actual-value (thunk-exp obj) (thunk-env obj))])
(set-mcar! obj 'evaluated-thunk)
(set-mcar! (mcdr obj) result)
(set-mcdr! (mcdr obj) '())
result)]
[(evaluated-thunk? obj)
(thunk-value obj)]
[else obj]))
(define (eval-if exp env)
(if (true? (actual-value (if-predicate exp) env))
(eval (if-consequent exp) env)
(eval (if-alternative exp) env)))
(define (eval-sequence exps env)
(cond [(last-exp? exps)
(eval (first-exp exps) env)]
[else
(eval (first-exp exps) env)
(eval-sequence (rest-exps exps) env)]))
(define (eval-assignment exp env)
(set-variable-value!
(assignment-variable exp)
(eval (assignment-value exp) env)
env)
'ok)
(define (eval-definition exp env)
(define-variable!
(definition-variable exp)
(eval (definition-value exp) env)
env)
'ok)
; Numbers and strings self evaluate
(define (self-evaluating? exp)
(cond [(number? exp) true]
[(string? exp) true]
[else false]))
(define (variable? exp)
(symbol? exp))
(define (tagged-list? exp tag)
(if (mpair? exp)
(eq? (mcar exp) tag)
false))
(define (quoted? exp)
(tagged-list? exp 'quote))
(define (text-of-quotation exp)
(mcar (mcdr exp)))
(define (assignment? exp)
(tagged-list? exp 'set!))
(define (assignment-variable exp)
(mcar (mcdr exp)))
(define (assignment-value exp)
(mcar (mcdr (mcdr exp))))
(define (definition? exp)
(tagged-list? exp 'define))
(define (definition-variable exp)
(if (symbol? (mcar (mcdr exp)))
(mcar (mcdr exp))
(mcar (mcar (mcdr exp)))))
(define (definition-value exp)
(if (symbol? (mcar (mcdr exp)))
(mcar (mcdr (mcdr exp)))
(make-lambda
(mcdr (mcar (mcdr exp)))
(mcdr (mcdr exp)))))
(define (lambda? exp)
(tagged-list? exp 'lambda))
(define (lambda-parameters exp)
(mcar (mcdr exp)))
(define (lambda-body exp)
(mcdr (mcdr exp)))
(define (make-lambda parameters body)
(mcons 'lambda (mcons parameters body)))
(define (if? exp) (tagged-list? exp 'if))
(define (if-predicate exp) (mcar (mcdr exp)))
(define (if-consequent exp) (mcar (mcdr (mcdr exp))))
(define (if-alternative exp)
(if (not (null? (mcdr (mcdr (mcdr exp)))))
(mcar (mcdr (mcdr (mcdr exp))))
'false))
(define (make-if predicate consequent alternative)
(mlist 'if predicate consequent alternative))
(define (begin? exp)
(tagged-list? exp 'begin))
(define (begin-actions exp) (mcdr exp))
(define (last-exp? seq) (null? (mcdr seq)))
(define (first-exp seq) (mcar seq))
(define (rest-exps seq) (mcdr seq))
(define (sequence->exp seq)
(cond [(null? seq) seq]
[(last-exp? seq) (first-exp seq)]
[else (make-begin seq)]))
(define (make-begin seq) (cons 'begin seq))
(define (application? exp) (mpair? exp))
(define (operator exp) (mcar exp))
(define (operands exp) (mcdr exp))
(define (no-operands? ops) (null? ops))
(define (first-operand ops) (mcar ops))
(define (rest-operands ops) (mcdr ops))
(define (cond? exp)
(tagged-list? exp 'cond))
(define (cond-clauses exp) (mcdr exp))
(define (cond-else-clause? clause)
(eq? (cond-predicate clause) 'else))
(define (cond-predicate clause)
(mcar clause))
(define (cond-actions clause)
(mcdr clause))
(define (cond->if exp)
(expand-clauses (cond-clauses exp)))
(define (expand-clauses clauses)
(if (null? clauses)
'false ;no else clause
(let ([first (mcar clauses)]
[rest (mcdr clauses)])
(if (cond-else-clause? first)
(if (null? rest)
(sequence->exp (cond-actions first))
(error "ELSE clause isn't last: COND->IF" clauses))
(make-if (cond-predicate first)
(sequence->exp (cond-actions first))
(expand-clauses rest))))))
(define (true? x)
(not (eq? x false)))
(define (make-procedure parameters body env)
(mlist 'procedure parameters body env))
(define (compound-procedure? p)
(tagged-list? p 'procedure))
(define (procedure-parameters p) (mcar (mcdr p)))
(define (procedure-body p) (mcar (mcdr (mcdr p))))
(define (procedure-environment p) (mcar (mcdr (mcdr (mcdr p)))))
(define (enclosing-environment env) (mcdr env))
(define (first-frame env) (mcar env))
(define the-empty-environment 'the-empty-environment)
(define (make-frame variables values)
(mcons variables values))
(define (frame-variables frame) (mcar frame))
(define (frame-values frame) (mcdr frame))
(define (add-binding-to-frame! var val frame)
(set-mcar! frame (mcons var (frame-variables frame)))
(set-mcdr! frame (mcons val (frame-values frame))))
(define (extend-environment vars vals base-env)
(if (= (mlength vars) (mlength vals))
(mcons (make-frame vars vals) base-env)
(if (< (mlength vars) (mlength vals))
(error "Too many arguments supplied"
vars
vals)
(error "Too few arguments supplied"
vars
vals))))
(define (lookup-variable-value var env)
(define (env-loop env)
(define (scan vars vals)
(cond [(null? vars) (env-loop (enclosing-environment env))]
[(eq? var (mcar vars))(mcar vals)]
[else (scan (mcdr vars) (mcdr vals))]))
(if (eq? env the-empty-environment)
(error "Unbound variable" var)
(let ([frame (first-frame env)])
(scan (frame-variables frame)
(frame-values frame)))))
(env-loop env))
(define (set-variable-value! var val env)
(define (env-loop env)
(define (scan vars vals)
(cond [(null? vars) (env-loop (enclosing-environment env))]
[(eq? var (mcar vars)) (set-mcar! vals val)]
[else (scan (mcdr vars) (mcdr vals))]))
(if (eq? env the-empty-environment)
(error "Unbound variable: SET!" var)
(let ([frame (first-frame env)])
(scan (frame-variables frame)
(frame-values frame)))))
(env-loop env))
(define (define-variable! var val env)
(let ([frame (first-frame env)])
(define (scan vars vals)
(cond [(null? vars) (add-binding-to-frame! var val frame)]
[(eq? var (mcar vars)) (set-mcar! vals val)]
[else (scan (mcdr vars) (mcdr vals))]))
(scan (frame-variables frame)
(frame-values frame))))
(define (setup-environment)
(let ([initial-env (extend-environment
(primitive-procedure-names)
(primitive-procedure-objects)
the-empty-environment)])
(define-variable! 'true true initial-env)
(define-variable! 'false false initial-env)
initial-env))
(define (primitive-procedure? proc)
(tagged-list? proc 'primitive))
(define (primitive-implementation proc)
(mcdr proc))
(define primitive-procedures
(mlist
(mcons 'car car)
(mcons 'cdr cdr)
(mcons 'cons cons)
(mcons 'null? null?)
(mcons '+ +)
(mcons '- -)
(mcons '/ /)
(mcons '* *)
(mcons '= =)))
(define (primitive-procedure-names)
(mmap mcar primitive-procedures))
(define (primitive-procedure-objects)
(mmap
(lambda (proc)
(mcons 'primitive (mcdr proc)))
primitive-procedures))
(define (apply-primitive-procedure proc args)
(apply (primitive-implementation proc) (if (mpair? args) (mlist->list args) args)))
(define the-global-environment (setup-environment))
(define input-prompt ";;; M-Eval input:")
(define output-prompt ";;; M-Eval value:")
(define (list->mlist/deep input)
(mmap
(lambda (value)
(if (pair? value)
(list->mlist/deep value)
value))
(list->mlist input)))
(define (driver-loop)
(prompt-for-input input-prompt)
(let ([input (read)])
(let ([output (actual-value (list->mlist/deep input) the-global-environment)])
(announce-output output-prompt)
(user-print output)))
(driver-loop))
(define (prompt-for-input string)
(newline)
(newline)
(display string)
(newline))
(define (announce-output string)
(newline)
(display string)
(newline))
(define (user-print object)
(if (compound-procedure? object)
(display
(list 'compound-procedure
(procedure-parameters object)
(procedure-body object)
'<procedure-env>))
(display object)))
(driver-loop)