mir/apparmor
2
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor synced 2025-08-31 06:16:03 +00:00
Code Issues Releases Wiki Activity

Update CompilerImprovements

John Johansen
2021-01-28 11:03:00 +00:00
parent 0782355e10
commit 24bc57b9bf
1 changed files with 22 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
CompilerImprovements.md

@@ -4,31 +4,35 @@ performance over each release.
Areas for improvement
=====================
1. lexer - Front end duplicate include elimination
- per profile cache which files have already been included so they don't need to be processed multiple times
#. lexer - Front end duplicate include elimination
- per block cache which files have already been included so they don't need to be processed multiple times
2. Include parse caching
- cache include files post parsing so that they don't need to be re-parsed
3. Rule duplicate removal (only done on files)
#. file cache (memory for speed)
* Include parse caching
- cache include files post parsing so that they don't need to be re-parsed
* generic file cache
- cache files that have already been read so they don't need to be reread
- requires multiple profiles be processed by one compiler call to be effective (especially if duplicate include elimination is in effect)
#. Rule duplicate removal (only done on files)
- almost unneeded if fix for 1 is done
- requires: to full replace (so code can be dropped) need 1 and 4
4. aare -> pcre convertion
#. aare -> pcre convertion
- add native parsing of aare to the background eliminating this step
5. early tree node merging
#. early tree node merging
- as part of tree creation instead of blindly creating new nodes, merge into existing tree as parsing
- requires: 3
6. tree simplification
#. tree simplification
- store alt nodes is sorted vector
- store cat nodes in vector
- do tree factoring on vectors, does not require tree rewrites or left/right manipulations
- merge char nodes into charset nodes
- replace char nodes with charset node to remove duplicate code
7. dfa creation - currently creates sets and removes duplicates which means a lot of creation and freeing
#. dfa creation - currently creates sets and removes duplicates which means a lot of creation and freeing
- push anode and nnode split into expr tree for nullable, firstpos, lastpos and follow calculations
- removes need to split sets, reducing allocation of set just to split it apart and lookup its constituent anode and nnode sets
- push anode and nnode caching into expr tree so it can be used by nullable, firstpos, lastpos
@@ -41,15 +45,12 @@ Areas for improvement
- Eliminate nnode cache?
- its possible that between the nodevec merge cache and final combined nnode, anode cache the dedicated nnode cache is no longer benifical
8. comb compression
#. comb compression
- change to sliding window algorithm, to reduce set of slot comparisons done
- Note: diff compression also reduces the number of slot comparisons being done by reducing the number of transition
10. file cache
- cache files that have already been read so they don't need to be reread
- requires multiple profiles be processed by one compiler call to be effective (especially if duplicate include elimination is in effect)
11. parallel compile
#. parallel compile
- compile individual profiles in parallel - separating the compile at the profile level should be fairly easy
- requires removal of global state
- compile profile components in parallel (see partial compiles)
@@ -59,12 +60,13 @@ Areas for improvement
- tree optimization -> DFA construction -> minimization -> diff-encode -> comb compression
- since each stage is separate but dependent, separate threads could work on a given thread
12. partial compiles - shared/precompiled
#. partial compiles - shared/precompiled
- share and potentially cache partial compile components
- includes are a natural boundary to precompile
- Requires dfa set operations
Improvements per Release
I
mprovements per Release
========================
- 2.1 DFA introduced
@@ -76,6 +78,9 @@ Improvements per Release
- 2.8 diff encoding
- 2.9 remove node hashing, fixing full minimization
- 2.10 early accept node grouping, and removal of accept alt node trees
- 2.11 -j N parallel compiles,
- 2.13 per kernel cache files
- 3.0 --jobs=0, limit expr-simplification passes,
Performance Tests
=================