- All compilation stages now make use of the `Compiler` object
Compiler
- Added new exception type `CompilerException` complete with a sub-type enum, `CompilerError`
- `getConfig()` will now throw a `CompilerException` when a key is not found, rather than return false (which didn't work under different template types anyways)
- Implemented `hasConfig()` to check for the existence of a key in the configuration sub-system's key-value store
- The `Compiler` object now stores the `Token[] tokens` generated from the call to `doLex()`
- The `Compiler` object now stores the resulting container (`Module`) generated from the call to `doParse()`
- Set default symbol mapping technique to the hashmapper technique
- Implemented `dolex()` for performing tokenization, it will create and store a `Lexer` instance and the produced `Token[] tokens` into the `Compiler` object
- Added `getTokens()` to fetch the tokens generated by `doLex()`
- Implemented `doParse()`, `doTypeCheck()` and `doEmit()` in a similiar fashion to `doLex()`
- Implemented `getMdoule()` to get the container (`Module`) generated by `doParse()`
- Implemented `compile()` which calls `doLex()`, then `doParse()`, then `doTypeCheck()` and finally `doEmit()`
CodeEmitter
- The `CodeEmitter` constructor now takes in an instance of the chosen `SymbolMapper`
DGen
- Switched to the instance of the `mapper` inheited from the `CodeMapper` parent class for any `symbolMap` calls required
- Use the inherited `TypeChecker` instance and not an instance of it provided by `Context`
SymbolMapper
- Reworked this class into an abstract class which must have its children implement a `symbolMap(Entity)` interface, this provides us pluggable mapping techniques
HashMapper
- Moved hashing symbol-mapping technique into `HashMapper`
Lebanese
- Created a kind-of `SymbolMapper` which, unlike `HashMapper`, produces human-redable-yet-valid C symbols (by replacing the `.`'s with `_`'s)
TypeChecker
- Removed code for setting now-nonexistent `SymbolMapper.tc`
- Removed code for setting now-nonexistent `Context.tc`
Context
- Removed `static TypeChecker tc` field
- Fully integrated new `LexerException` system into `performLex()`
COmmand-line
- Updated `commands.d` to catch any `TError` and report the error back
Parser
- Updated unittests to use `LexerException`
Compiler
- Disabled unit tests for now as they cause errors
- `getLinearizedNodes()` returns the `DNode[]` of the linearization results
` `getTree()` returns the string representation of the dependency tree
- New linearization method which requires calling `performLinearization()` before `getLinearizedNodes()` or `getTree()`, if not then an exception is thrown
- Made `tree()` private
- `tree()` now takes in argument `ref DNode[]` of which it will read the linerization into instead of `static DNode[] poes`
- Added comment regarding Problem 5 of issue #41
- Removed `static DNode[] poes`
TypeCheck
- Added better comments
- Switched to new linearization methods for both module-level processing and function definition (`FuncData`) processing
- Removed any reference to `DNode.poes`
- Moved configuration sub-system to its own class `CompilerConfiguration`
- `getConfig()` and `setConfig()` are now templatised to generate, at compile-time, type-specific versions which will fetch and convert using the requested type
- Added some more default configuration parameters for DGen
DGen
- Implemented configuration checking for `genTabs()` and `emitEntryPoint()` (both which have the default config value of `true`)
- Added commented-out testing code (see issue #88)
- Added missing `typeChecker.beginCheck()` call to the TypChecker in `compile()`
- Print error on lexing error
- Removed old compilation code and replaced with an instantiation of a `Compiler` object and a corresponding call to `.compile()`
- Added comments and removed TODO for now complete feature on `parseTypedDeclaration()`
Dependency
- Added support for external variable symbols to `transform()` for declarations, assignments and variable expressions
Test cases
- Updated extern evar test case to new symbol name that wouldn't clash
- Added assignment usage and expression usage
- `parseTypedDeclaration()` will no longer consume the SEMICOLON after a vardec
- `parseTypedDeclaration()` will no longer consume theSEMICOLON after an assignment
- `parseName()` will expect a semicolon if the returned object from `parseTypedDeclaration()` is of type `Variable`. If of type `Function` then nothing will be expected
- Added `allowVarDec` and `allwFuncDef` to `parseTypedDeclaration()` as default arguments with `true` as their default values
- If `allowFuncDef` is true then parse a function
- If `allowVarDec` is `true` then allow a variable definition
- If `allowVarDec` is true but we have an assignment and `wantsBody` is false then throw an error
- NOTE: A fix for `parseTypedDeclaration()` is needed for this to work
Tests
- Updated `simple_extern.t` to include an `extern evar`
- Implemented type `Compiler` which wraps compilation all into one object which can have `compile()` called on it to perform the full sequence of tests
- Added new symbol types `EXTERN`, `EXTERN_EFUNC` and `EXTERN_EVAR` and related back-mappings
Parser
- `parseFuncDef()` now accepts a default argument (set to `true`) on whether to expect a body for a function or not, in the not case expect a semi-colon - this helps with extern support
- Likewise because `parseFuncDef(bool wantsBody = true)` is called by `parseTypedDeclaration()` we have added same argument to `parseTypedDeclaration(bool wantsBody = true)`
- Ensure we pass the parameter from `parseTypedDeclaration()` into `parseFuncDef(bool)` in function definition case
- Implemented `parseExtern()` for extern support
- `parse()` supports `SymbolType.EXTERN` now
Data
- Added `ExternStmt` to represent the parser node derived from a call to `parseExtern()`
- The `Entity` parser node type now has an `isExternal()` flag to know if the entity is marked for `extern` link time or TLang internal time (default)
Typechecker
- Implemented `processPseudoEntities(Container)` which loops through the given container and finds all extern statements and then extracts those nodes, parents them to the given container and marks them as external (pseudo-handling support)
- Added first call inside `beginCheck()` to be a call to `processPseudoEntities(modulle)`
Dependency
- Added useless no-op check for `ExternStmt` - it does nothing
DGen
- In `emitFunctionSignature()`, prepend the string `extern ` to the signatur if the given `Function` entity is marked as external (`isExternal()` is true)
- In `emitFunctionDefinitions()` do not emit a function body at all (or anything, no signature) if the `Function` is marked as external (`isExternal()` is true)
- Added entry point test for `simple_extern.t`
- Added `getReferType()` to `Pointer` type to refer the `Type` instance of the data being pointed to
DGen
- Fixed bug whereby pointer types were not being transformed correctly in typeTransform()`
- VariableParameter's now have their `Type` object extracted and type transformed as part of function definitions.
Tests cases
- Updated `simple_cast.t` to test new typeTransform() usage on VariableParameter (as stated above)
- Extract the `Variable`'s `Type` object and pass it into the instruction constructor
Instruction
- `VariableDeclaration` instruction now takes in an instance of `Type` upon construction
Dependency
- Fixed null pointer exception where Function did not have its `context` set
DGen
- Added `typeTransform(Type)` to transform the given types into the C equivalent
- Variable declarations use `typeTransform()` now
- Casting instructions use `typeTransform()` now
- Added `emitStdint()` to emit `#include<stdint.h>` as part of header in generated C code
- `generateSignature(Function)` now uses `typeTransform()` for the return type emit
- Implemented new instruction `CastedValueInstruction`
- Added import for `Type`
DGen
- Implemented primitive type casting code generation in `transform()`
- Added import for `Type` and `Primitive`
Parser
- Implemented `parseCast()` which is called by `parseExpression()`
Check
- Added new symbol type `SymbolType.CAST`
Expressions
- Implemented new expression type `CastedExpression`
Typechecker
- Implemented processing of `CastedExpression` in the expression section, along with correct type/instruction pushes and pops
Dependency
- Implemented dependency generation of `CastedExpression` in `expressionPass()`
Test cases
- Added test file `simple_cast.t`
- Added `getReferredType()` to `Pointer` to fetch the type of the data being referred to
Typechecker
- Unary operator `STAR` now will check popped type, ensure it is a pointer, then push the type of the referred-to data
Test cases
- Updated the `simple_pointer.t` test case to do pointer dereferencing
DGen
- Updated entry point testing code for the pointer test `simple_pointer.t`
- Added new instruction `DiscardInstruction`
DGen
- Added ability to transform `DiscardInstruction`
Parser
- Implemented `parseDiscard()`
- Removed a TODO
- Added a unittest testing the new `parseDiscard()`
- Typo fixes here and there in unittests
Data
- Added new parser node `DiscardStatement`
Typechecker
- Added codegen for `DiscardStatement` type
Dependency
- Added dependency processing for `DiscardStatement` type
Tests
- Added new test case `simple_discard.t`
- Added support for n-ary pointer types in function parameters
- Added a test case for the above
- Updated current pointer test to test function type n-ary as well (return type)
- Added `getOperator()` and `getOperand()` methods to `UnaryOpInstr`
- Added new instruction `PointerDereferenceAssignmentInstruction` for pointer support
DGen
- Updated `transform()` to emit code for instruction type `UnaryOpInstr`
- Updated `transform()` to emit code for instruction type `PointerDereferenceAssignmentInstruction`
- Added testing emit code in `emitEntryPoint()` for pointer testing
Parser
- Updated `parseName()` to trigger `parseTypedDeclaration()` on occurene of `SymbolType.STAR` (for pointer type declarations)
- Added pointer-type support for function parameters (so far only single) in `parseFuncDef()`
- `parseExpression()` terminates on occurence of a single `=` (ASSIGN) operator
- Declaring of pointers of any depth implemented in `parseTypedDeclaration()`
- Added support for pointer dereferncing assignments with the addition of `parseDerefAssignment()`
- `parseStatement()` will now call `parseDerefAssignment()` on occurence of a `SymbolType.STAR`
- Added a unittest for testing pointers
- Finished unittest for for loops
Check
- Added backmapping for `SymbolType.ASSIGN` -> `&`
Data
- Added new parser node type `PointerDereferenceAssignment` for pointer support in the parser
TypeChecker
- Because function parameters are type che cked upon function call I had to add typechecking code for pointer support in the `UnaryOperatorExpression` case
- Added code generation support for `PointerDereferenceAssignment` type
Dependency
- Added support for `PointerDereferenceAssignment` type (pointer support) to `generalStatement()`
Tests
- Added pointer test `simple_pointer.t`
- Added a new instruction, `ForLoop`, which contains a pre-run Instruction and a `Branch` instruction, coupled with some flags
DGen
- Added a TODO for WhileLoops (we need to implement do-while loops)
- Implemented C code emitting in `emit()` for `ForLoop` instruction
Check
- Added missing back-mapping for `SymbolType.SMALLER_THAN`
Data
- Added new parser node type `ForLoop`
Parser
- Fixed typo in `parseWhile()`
- Implemented `parseDoWhile()` for do-while loops
- Implemented `parseFor()` for for-loops
- Implemented `parseStatement()` for singular statement parsing
- `parseStatement()` can now have the terminating symbol specified, defaults to `SymbolType.SEMICOLON`
- `parseName()` and `parseAssignment()` now also accept a terminating symbol parameter as per `parseStatement()`'s behavior
- `parseBody()` now makes multiple calls to `parseStatement()` for singular Statement parsing (dead code below still to be removed)
- Removed commented-out unittests
- Unittests that read from files now have the file source code embedded
- Added unit test for while loops, for-loops (unfinished) and some other smaller language constructs (roughly 70% coverage)
TypeChecker (CodeGen)
- Do-while loops will fail if used (for now)
- Added for-loop code generation
Dependency
- Implemented `generalStatement()` for statement processing
- `generalPass()` now makes calls to `generalStatement()`
Tests
- Added `simple_for_loops.t` to test for-loops
- Added `simple_do_while.t` to test do-while loops
- Added new instruction `WhileLoopInstruction`
DGen
- Added support for emitting while-loops (so far just plain while loops) (`WhileLoopInstruction` in `emit()`)
- Added baked-in entry point testing code for while loops in `emitEntryPoint()`
Parsing
- Added missing plumbing for while loop parser nodes in `parseWhile()`
Data
- Fixed some typos
- Removed dead/unused "deps" code from `Entity`
- Added some documentation comments
- Added `WhileLoop` type for parser nodes
TypeChecker
- Removed TODO in comment for already-implemented/completed if-statements
- Added while-loop code generation support (only while-loops, no do-whiles)
Dependency
- Added while-loop dependency generation support (so far only while-loops, no do-whiles)
Tests
- Added new test case `simple_while.t` for testing while loops
- Fixed missing flushing for issue #65 (see "Flushing fix ✅")
- Added unit test for flushing fix
VariableDeclaration (Instruction)
- Added support for the embedding of a VariableAssignmentInstr inside (added a getter too) (a part of issue #66)
- Conditional support for if statements: Added two new instructions (IfStatementInstruction and BranchInstruction). See issue #64
DGen
- Added depth increment/decrement on enter/leave scope of `transform()`
- Correct tabbing for nested if-statements using new method `genTabs(ulong)` (which uses the above mechanism). Makes code emitted for if statements (issue #64) look nicer.
- Updated VariableDeclarations (with assignments) handling in `transform()` in the manner similar to BinOpInstr (see issue #66)
- Added a TODO for formatting BinOpInstr's `transform()` a little more aesthetically nicer
- Added code emitting support for if statements (the `IfStatementInstruction` instruction) (see issue #64)
- Updated `emitEntryPoint()` to only emit testing C code for the correct input test file
Parser
- `parseIf()` now returns an instance of IfStatement which couples multiple `Branch` objects consisting of `Statement[]` and `Expression`
- Ensured that each `Statement` of the generated `Statement[]` from `parseBody()` for a given `Branch` is parented to said Branch using `parentToContainer()`
- Ensured each generated `Branch` in `Branch[]` is parented to the generated `IfStatement` using `parentToContainer()`
- `parseBody()` now adds to its `Statement[]` build-up array the generated `IfStatement` from the call to `parseIf()`
Check
- Added support for back-mapping `SymbolType.EQUALS` to `getCharacter(SymbolType)`
Data
- Added `Branch` parser node which is a Container for body statements (`Statement[]`)
- Added `IfStatement` parser node which is a Container of `Statement[]` which are actually `Branch[]`
TypeChecker
- Moved import for `reverse` to top of module
- Implemented `tailPopInstr()` method which will pop from the back of the `codeQueue` "scratchpad"
- Fixes handling of `StaticVariableDeclaration` and `VariableAssignmentNode` (fixes issue #66)
- Added handling for IfStatement entities (if statement support #64)
Resolution
- Added a debug statement to `resolveUp(Container, string)` to print out the container to lookup from and the name being looked up
Dependency
- Added a default `toString()` to the DNode class which prints `[DNode: <entity toString()]`
- Added a TODO and debug print related to issues #9
- Disabled InitScope.STATIC check for now as it caused issues with if statement parsing (probably due to VIRTUAL being default and therefore skipping if statment processing) - issue #69
- Cleaned up handling of Entity type `Variable` (variable declarations) - removed repeated code
- Undid the VarAss->(depends on)->VarDec, reverted back to VarDec->(depends on)->VarAss, fixed by #66 (and closes it and #11)
- Added support for `IfStatement` (if statements) in `generalPass(Container, Context)`
Test cases
- Added new test case testing nested if statements (`nested_conditions.t`)
- Added another test case for if statements, `simple_conditions.t`
- Added new instruction type `ReturnInstruction`
Data
- Adjusted return statement parser node weighting to 2 (makes it on the same level as normal body statements)
Dependency
- Added dependency generation for return statements
- Removed old commented-out code in the function-definition generation section of `generalPass()`
TypeChecker/Codegen
- Added code generation for return statements
DGen
- Added code emitting for return statements (`ReturnInstruction`)
Test cases
- Updated test case `simple_functions.t` to test return statements
- Cleaned up emit() method
- New test case hashes inserted for entryPoint test code
Test cases
- Updated `simple_function_decls.t` to be exactly that, ONLY declarations with basic bodies
- Migrated advanced function usage testing code to a new unit test: `simple_functions.t`
DGen
- Updated function parameter symbol lookup to use new `symbolLookup9Entity)` mechanism
Test cases
- Updated test case `simple_function_decls.t` to use arguments referencing for tests
- Bug fix (#57) - Undid hack (of #46) for Context setting of VariableStdAloneAssignments
- Bug fix (#54) in VariableExpression whereby global lookups failed because they used `resolveWithin()` and not `resolveBest()`
Resolution
- Implemented `generateNameBest(Entity)` which can generate the full absolute path of the given Entity
- Added debug statements to `isDescendant(Container, Entity)`
- Added a TODO for when isDescendant fails, the asserts should be removed and placed there
Mapper
- The `symbolLookup()` method now takes in only the Entity and provides a hash (by making use of `generateNameBest()`)
DGen
- Switched to using the new `symbolLookup(Entity)` in `transform()` wherever it was being used
Test cases
- Updated test case `simple_function_decls.t` to use a global variable reference in a VariableExpression to test the fix for #54
- Make the name of the function const and public for FuncCallInstr
CodeEmitter
- Added methods `getCursor()`, `getSelectedQueueLength()` and `getQueueLength()`
- Removed old queue-specific methods
DGen
- Added emitting for FuncCallInstr instruction (function call support)
- Now emit globals first BEFORE function definitions
- Added debug prints per instruction to know what instruction is currently being transform()'d
- After emitting sections add newlines between each to make for neater C code
- `emitEntryPoint()` now adds a test for `simple_function_decls.t` (This should be removed soon)
- Removed incorrect TODO in `finalize()`
Dependency
- Make the `nodePool` static, to ensure pooling carries over across multiple `DNodeGenerator` instances
- Fixed handling of function calls in `expressionPass()` - do NOT add a so-called `FunctionDefNode` (remember functions are defined by `addFuncDef()`)
- Set the Context of standalone variable assignments to the Context of the Variable entity representing the variable being assigned to
TypeChecker
- Assign the Context object stored in the `FunctionCall` statement to the `FuncCallInstr`
Test cases
- Updated test case `simple_function_decls.t`
- Added newline to release info print
- Fixed module docstring
Commands
- Added new command-line options: `syntaxcheck`, `typecheck`
- Added todo to `help` command
- Re-ordered commands for order of appearance in help text
Compiler
- Added docstring to `beginCompilation(string[])` function
Mapper
- Added debug print of the Container being used for the symbol lookup
CodeEmitter
- Re-worked CodeEmitter class to use a single so-called "selected queue"
- Added methods to move back and forth between said "selected queue", get the length, etc.
- Remove old queue-specific methods
DGen
- Use the new CodeEmitter "selected-queue" functionality
- Emit function definitions now supported
Exceptions
- Added this keyword
Check
- Added support for SymbolTYpe.OCURLY and SymbolType.CCURLY to `getCharacter(SymbolType)`
Data
- Added a `hasParams()` method to the Function entity type
TypeChecker
- Added support for emitting function definitions (required DNode.poes = [] (cleaning), codeQueue cleaning etc.)
- Added `getInitQueue()` method to make a copy of the current "scratchpad" `codeQueue`
- Build up a copy of the global queue now (make a copy similiar to what we did for `getInitQueue()` but inline)
- Added a debug print
Dependency
- Added a FIXME note for issue #46
- Added a TODO relating to `static DNode[] poes`
Test cases
- Added test case `simple_function_decls.t` to test function definition code emit
- Updated test case `simple_variables.t` to note that the T code generates invalid C code
README
- Build instructions now generate coverage files (`.lst`s)
- Updated link to documentation
- Added support for return statements
- Fixed up SyntaxError to display informative messages
- `expect(SymbolType, Token)` will now use the new SyntaxError constructor
Data
- Added `ReturnStmt` type for return statement support
- Implemented code emit for variable expressions (fetching their values)
----
Test cases
- Updated test case `simple_variables.t` to be able to test the newly implemented `FetchValueInstr` code emit
- Removed redundant imports
Dependency
- Set the context for the `VariableAssignmentStdAlone` entity
- Removed words that show I have the mental maturity of a 12 year old
----
Test cases
- Updated `simple_variables_decls_ass.t`
- Updated `simples_variables.t`
- Removed duplicate instance of `VariableAssignmentInstr`
- Set the context of the created `VariableAssignmentInstr` to be that of the `VariableAssignmentStdAlone` entity
- All operations regarding moving through (iterating over) instructions are now to be done via the instructions provided by CodeEmitter, moveback, moveforward etc.
DGen
- Re-worked the system to use new CodeEmitter semantics
- Variable assignments in declarations are oneshot now which makes it more compact, semantically better and also valid C
---
Tests
- Updated the `simple_variables.t` test case to have many binary operations chained
- Removed all `emit()` methods (this is now done in the language-specific emitter - DGen)
CoreEmitter
- Added docstrings
- Added required `transform(Instruction)` method which must transform each provided Instruction into a string (a.k.a. do the actual emit()-ting)
DGen
- Migrated C-emit code into the overrided `transform(Instruction)` method
- VariableAssignment entity now has its Context object set to the current Context (of the Variable being declared) (so this is a declare assignment case only (so far)) (fixes#36)
TypeChecker
- Extract the Context object from the VariableAssignment entity and then set it as the Context for the VariableAssigmnetInstr instruction (fixes#36)
VariableAssigmnentInstr
- The `emit()` method will now emit the assignment code
Check
- Added `getCharacter(SymbolType)` which maps a SymbolType to a maths operator (WIP)
- Implemented the correct emit for this instruction (just emits the string version of the ulong `data`
BinOpInstr
- Emit is almost correct, just missing a mapping from SymbolType to the maths operator required
- Added new SymbolMapper class with a static method which will take a Container and an entity name, resolve the Entity and then generate the hash of the absolute path to said entity and return this as the `symbol name`
TypeChecker
- Set the static field to refer to this instance of the TypeChecker (in the SymbolMapper class)
VariableDeclaration
- Use the `symbolLookup()` method to transform the name
- Added `finalize()` method, this is to be called whenever the emitting is done and a compiler is to be called on the code
DGen
- Implemented a gcc-based finalization method
- Added `emitEntryPoint()` to emit a main() function which is to be called in libc's `_start` symbol
VariableDeclaration
- Added note we may need a symbol table lookup system actually, rather than just a simple `symbolRename`
Compiler
- Call the `finalize()` method on the DGen code emitter
----
Test cases
- Added `simple_variables_only_decs.t` to test code generation
- Added a method which will replace the `.`s in a symbol with underscores
Instruction
- The symbol names need to be valid in C so let's just make it a rule they will only have _'s (underscores) to replace any characters like `.`s
- Pass in the type of the variable being declared to the `VariableDeclaration` instruction
VariableDeclaration
- Emitted code now contains the type of the variable being declared
- Added `getInitQueue()`
- Removed the `beginEmit(initQueue, codeQueue)` method (as this is done for us in compiler.d
CodeEmitter
- Extract init queue as well
This is a queue where all things such as:
- Class static initializations
- Module initializations (maybe)
- Struct static initializations
will go.
Added `addInit()` along `SList initQueue` to support such a queue.
- Whenever a ClassStaticNode is come across it is added to this queue (at the back - so as to maintain allocation order, not that allocation would matter - only initialization (which is already working))
- The VarAssDNode processor on typechecker now adds to the back orf the code queue
- We removed all weird swapping code in typechecker
- Dependency wise, a variable declaration is depended-UPON its variable assignment which in turn a module depends ON.
- In the case of no assignments we simpyl make the module depend on the variable declaration dnode directly
- Added new test case to show this all
- The test case was broken -_-
- Full-path traversal now implemented somewhat, along with checks on the container if they require static initialization (such as for Classes)
Full path evaluation update
If the local segment is a Class-type-of Container then run
a class static allocation on it and make the current dnode
dependent on that ClassStaticAllocation dnode returned from
classStaticAllocate().
- Disabled seperate generate() calls to FuncDefs in core.d for typechecking
- Added `saveFunctionDefinitionNode()` and `retrieveFunctionDefinitionNode()` to core.d for dependency
- Added a generalPass() call (and fixed it to skip on things != STATIC for InitScope - temporary fix)
- Added test case to test with
