mir/apparmor
2
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor synced 2025-09-02 15:25:27 +00:00
Code Issues Releases Wiki Activity

Update CompilerImprovements

John Johansen
2021-01-28 11:03:00 +00:00
parent 0782355e10
commit 24bc57b9bf
1 changed files with 22 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
CompilerImprovements.md

@@ -4,31 +4,35 @@ performance over each release.
Areas for improvement Areas for improvement
===================== =====================
1. lexer - Front end duplicate include elimination #. lexer - Front end duplicate include elimination
- per profile cache which files have already been included so they don't need to be processed multiple times - per block cache which files have already been included so they don't need to be processed multiple times
2. Include parse caching #. file cache (memory for speed)
- cache include files post parsing so that they don't need to be re-parsed * Include parse caching
- cache include files post parsing so that they don't need to be re-parsed
3. Rule duplicate removal (only done on files) * generic file cache
- cache files that have already been read so they don't need to be reread
- requires multiple profiles be processed by one compiler call to be effective (especially if duplicate include elimination is in effect)
#. Rule duplicate removal (only done on files)
- almost unneeded if fix for 1 is done - almost unneeded if fix for 1 is done
- requires: to full replace (so code can be dropped) need 1 and 4 - requires: to full replace (so code can be dropped) need 1 and 4
4. aare -> pcre convertion #. aare -> pcre convertion
- add native parsing of aare to the background eliminating this step - add native parsing of aare to the background eliminating this step
5. early tree node merging #. early tree node merging
- as part of tree creation instead of blindly creating new nodes, merge into existing tree as parsing - as part of tree creation instead of blindly creating new nodes, merge into existing tree as parsing
- requires: 3 - requires: 3
6. tree simplification #. tree simplification
- store alt nodes is sorted vector - store alt nodes is sorted vector
- store cat nodes in vector - store cat nodes in vector
- do tree factoring on vectors, does not require tree rewrites or left/right manipulations - do tree factoring on vectors, does not require tree rewrites or left/right manipulations
- merge char nodes into charset nodes - merge char nodes into charset nodes
- replace char nodes with charset node to remove duplicate code - replace char nodes with charset node to remove duplicate code
7. dfa creation - currently creates sets and removes duplicates which means a lot of creation and freeing #. dfa creation - currently creates sets and removes duplicates which means a lot of creation and freeing
- push anode and nnode split into expr tree for nullable, firstpos, lastpos and follow calculations - push anode and nnode split into expr tree for nullable, firstpos, lastpos and follow calculations
- removes need to split sets, reducing allocation of set just to split it apart and lookup its constituent anode and nnode sets - removes need to split sets, reducing allocation of set just to split it apart and lookup its constituent anode and nnode sets
- push anode and nnode caching into expr tree so it can be used by nullable, firstpos, lastpos - push anode and nnode caching into expr tree so it can be used by nullable, firstpos, lastpos
@@ -41,15 +45,12 @@ Areas for improvement
- Eliminate nnode cache? - Eliminate nnode cache?
- its possible that between the nodevec merge cache and final combined nnode, anode cache the dedicated nnode cache is no longer benifical - its possible that between the nodevec merge cache and final combined nnode, anode cache the dedicated nnode cache is no longer benifical
8. comb compression #. comb compression
- change to sliding window algorithm, to reduce set of slot comparisons done - change to sliding window algorithm, to reduce set of slot comparisons done
- Note: diff compression also reduces the number of slot comparisons being done by reducing the number of transition - Note: diff compression also reduces the number of slot comparisons being done by reducing the number of transition
10. file cache
- cache files that have already been read so they don't need to be reread
- requires multiple profiles be processed by one compiler call to be effective (especially if duplicate include elimination is in effect)
11. parallel compile #. parallel compile
- compile individual profiles in parallel - separating the compile at the profile level should be fairly easy - compile individual profiles in parallel - separating the compile at the profile level should be fairly easy
- requires removal of global state - requires removal of global state
- compile profile components in parallel (see partial compiles) - compile profile components in parallel (see partial compiles)
@@ -59,12 +60,13 @@ Areas for improvement
- tree optimization -> DFA construction -> minimization -> diff-encode -> comb compression - tree optimization -> DFA construction -> minimization -> diff-encode -> comb compression
- since each stage is separate but dependent, separate threads could work on a given thread - since each stage is separate but dependent, separate threads could work on a given thread
12. partial compiles - shared/precompiled #. partial compiles - shared/precompiled
- share and potentially cache partial compile components - share and potentially cache partial compile components
- includes are a natural boundary to precompile - includes are a natural boundary to precompile
- Requires dfa set operations - Requires dfa set operations
Improvements per Release I
mprovements per Release
======================== ========================
- 2.1 DFA introduced - 2.1 DFA introduced
@@ -76,6 +78,9 @@ Improvements per Release
- 2.8 diff encoding - 2.8 diff encoding
- 2.9 remove node hashing, fixing full minimization - 2.9 remove node hashing, fixing full minimization
- 2.10 early accept node grouping, and removal of accept alt node trees - 2.10 early accept node grouping, and removal of accept alt node trees
- 2.11 -j N parallel compiles,
- 2.13 per kernel cache files
- 3.0 --jobs=0, limit expr-simplification passes,
Performance Tests Performance Tests
================= =================