Architectural Notes

[sfkleach]

• Add notes on the architecture of the reference implementation
• Explain
  • Parsing entire input
  • Threading the tokens
  • Reserved word discovery
  • Dealing with gluing and line-breaks as soft semi-colons
• Hand-crafted tokeniser to deal with:
  • Multi-line strings
  • Interpolated strings
  • Gluing
  • Line breaks
• Discovery of reserved words
• Hand-crafted parser based on recursive descent with operator precedence
• Key idea: form context vs expression context