 |
Ansel 0.0
A darktable fork - bloat + design vision
|
|
Ansel 0.0
A darktable fork - bloat + design vision
|
Functions | |
| load_submodule_exclusions (repo_root=".") | |
| excluded_globs () | |
| is_production_file (path) | |
| is_excluded (path) | |
| run (cmd, **kw) | |
| collect_symbols (db_path) | |
| include_edges (db_path) | |
| source_include_edges (repo_root, source_dir="src") | |
| collect_includers (edges) | |
| collect_reach (edges) | |
| _modularity (adj_w, degree, two_m, partition) | |
| _louvain (adj_w, degree, two_m, passes=12) | |
| collect_modularity (edges, source_dir="src") | |
| collect_selfcontained (path) | |
| collect_docs (db_path) | |
| collect_git (repo_root, days, per_file_ccn, max_files_per_commit=20) | |
| module_of (path, source_dir="src") | |
| strongly_connected (nodes, succ) | |
| feedback_arc_order (edges) | |
| derive_layering (edges, label) | |
| compute_stability (mod_edges) | |
| collect_layering (edges, source_dir="src") | |
| collect_god_header (edges) | |
| collect_ccn (source_dir) | |
| collect_cppcheck (source_dir, jobs) | |
| collect_clang_tidy (log_path) | |
| collect_cloc (source_dir) | |
| md_table (headers, rows, aligns=None) | |
| build_markdown (project, data) | |
| main () | |
Variables | |
| list | EXCLUDED_DIR_PARTS |
| tuple | SOURCE_SUFFIXES = (".c", ".cc", ".cpp", ".cxx", ".h", ".hpp", ".hxx", ".m", ".mm") |
| PRODUCTION_LANGUAGES | |
| CLANG_TIDY_LINE | |
Build the "code health" panel published alongside the Doxygen API docs.
The panel answers one question: how manageable is this codebase, in numbers that
mean the same thing on darktable and on Ansel. It is generated identically in both
repositories so the two published sites can be read side by side.
Inputs, all optional except the first — a missing tool degrades its own section to
"not available" instead of failing the build:
doc/api/sqlite3/doxygen_sqlite3.db Doxygen's own symbol table (GENERATE_SQLITE3),
produced by a fast first Doxygen pass. Gives
symbols per file and the include graph.
lizard cyclomatic complexity (CCN) per function.
cppcheck static analysis without needing a build.
<clang-tidy report>.json/.txt clang-tidy findings, when a separate job that
can produce compile_commands.json has run.
Outputs:
doc/code-health.md a Doxygen page (picked up by INPUT, themed, searchable)
doc/code-health.json the same numbers, machine-readable, for cross-repo diffing
Usage:
python3 tools/code_health.py --project darktable --source-dir src \\
[--db doc/api/sqlite3/doxygen_sqlite3.db] [--clang-tidy-log FILE]
|
protected |
Louvain community detection, first phase iterated to convergence. Implemented here rather than pulled in, so the panel keeps needing nothing but python3 on the runner. Only the local-moving phase is used - repeated to a fixed point - which is enough to establish whether a better grouping than the directory layout exists, without the graph-coarsening phase's bookkeeping.
Definition at line 421 of file code_health.py.
Referenced by collect_modularity().
|
protected |
Newman modularity Q of one partition of a weighted undirected graph.
Definition at line 407 of file code_health.py.
References L.
Referenced by collect_modularity().
| code_health.build_markdown | ( | project, | |
| data | |||
| ) |
Definition at line 1312 of file code_health.py.
References A, items, L, max, and md_table().
Referenced by main().
| code_health.collect_ccn | ( | source_dir | ) |
Per-function cyclomatic complexity, via lizard. The distribution matters more than the total: a codebase's maintenance cost lives in its tail, not its mean, so the thresholds below are reported as counts.
Definition at line 1061 of file code_health.py.
References excluded_globs(), is_excluded(), L, max, min, and run().
Referenced by main().
| code_health.collect_clang_tidy | ( | log_path | ) |
Aggregate a clang-tidy run's console log by check name. Deliberately parses the log rather than running clang-tidy: producing compile_commands.json needs a configured build tree and the project's full dependency set, which does not belong in the docs job.
Definition at line 1202 of file code_health.py.
References is_excluded(), and L.
Referenced by main().
| code_health.collect_cloc | ( | source_dir | ) |
Lines of code, counted per file and filtered with this module's own predicate. cloc's --not-match-d is NOT used to drop vendored code: depending on the cloc version it matches a single path component rather than a subtree, so src/external/rawspeed/src/... survives a --not-match-d on "external". That went unnoticed locally and inflated the published figures to 1,012,392 lines against the real 331,243. Counting --by-file and filtering through is_excluded() is the only way this section agrees with every other section of the panel.
Definition at line 1252 of file code_health.py.
References is_excluded(), L, and run().
Referenced by main().
| code_health.collect_cppcheck | ( | source_dir, | |
| jobs | |||
| ) |
cppcheck findings by severity and by rule id. cppcheck is used rather than clang-tidy for the always-on panel because it needs no compile_commands.json, so it runs in the docs job on both repositories under identical conditions. clang-tidy findings, which need a configured build tree, are folded in from --clang-tidy-log when a job that can produce one has run.
Definition at line 1143 of file code_health.py.
References is_excluded(), and L.
Referenced by main().
| code_health.collect_docs | ( | db_path | ) |
How much of the API carries any documentation at all. Doxygen records a brief and a detailed description per symbol, so the tree's own documentation coverage is a query rather than an estimate. Counted over the same production files as everything else.
Definition at line 551 of file code_health.py.
References is_excluded(), L, and max.
Referenced by main().
| code_health.collect_git | ( | repo_root, | |
| days, | |||
| per_file_ccn, | |||
max_files_per_commit = 20 |
|||
| ) |
Evolution metrics: churn, hotspots, change coupling and ownership.
Process metrics predict defects better than static complexity does - complex code
nobody touches is harmless, complex code changed weekly is where the bugs are - and
none of the rest of this panel can see them, because they are not a property of the
code as it stands but of how it got there.
hotspot revisions x cyclomatic complexity. The prioritisation metric:
what to refactor first, rather than what is merely large.
change coupling files that keep changing together in the same commit. Some of
those pairs have no include edge between them at all, which is
coupling no static analysis can find.
ownership authors per file. Concentration is not automatically good or
bad - one author means fast decisions and a bus factor of one.
Commits touching more than max_files_per_commit production files are excluded from
the coupling counts only: a sweeping rename couples everything it touches to
everything else, which is an artefact of the commit rather than of the code. They
still count towards churn and ownership.
Definition at line 601 of file code_health.py.
References is_excluded(), L, max, min, partition(), and run().
Referenced by main().
| code_health.collect_god_header | ( | edges | ) |
Who includes the application-global header. darktable's src/common/darktable.h and Ansel's src/darktable.h are the same file by descent. The number that matters is how many HEADERS include it: a .c doing so is a choice local to that file, a .h doing so pushes the whole application into every file downstream of it.
Definition at line 1030 of file code_health.py.
References L.
Referenced by main().
| code_health.collect_includers | ( | edges | ) |
How many files include each header, directly. This is the number behind the "included by" graphs: the fan-in of a header, and the single clearest measure of how entangled a codebase's headers are.
Definition at line 275 of file code_health.py.
References L.
Referenced by main().
| code_health.collect_layering | ( | edges, | |
source_dir = "src" |
|||
| ) |
Dependency cycles and derived layering, at file and at directory level. Everything here comes from the include graph. Nothing is declared. Cycles are the objective part: if A depends on B and B on A, no layering of the two exists, whatever anyone believes. Reported as strongly connected components. The derived ordering is the graduated part: order the units so as few includes as possible point backwards, and report what still does. Those edges are the minimum set of dependencies that would have to go for a layering to exist at all. The FILE graph is the primary unit, because it presumes nothing about how the tree is organised - it does not need directories to be modules, which in darktable they are not. The directory graph is reported too, as an aggregation over what are really drawers rather than modules.
Definition at line 950 of file code_health.py.
References compute_stability(), derive_layering(), feedback_arc_order(), L, max, module_of(), and strongly_connected().
Referenced by main().
| code_health.collect_modularity | ( | edges, | |
source_dir = "src" |
|||
| ) |
Does the folder layout correspond to how the code is actually coupled? The directories are treated as a proposed partition of the dependency graph and scored with Newman modularity Q - the share of edges falling inside groups, minus what random wiring of the same degrees would produce. Then a partition is derived from the graph itself with Louvain and scored the same way. The GAP between the two is the number that matters. If directories really were modules, grouping by directory would be near-optimal and the gap would be small. A large gap means the folders are drawers: the code clusters, but not along the lines the tree is filed under. Also reported without any clustering at all: the share of includes that stay inside their own directory, which is the same question asked bluntly.
Definition at line 455 of file code_health.py.
References _louvain(), _modularity(), L, max, and module_of().
Referenced by main().
| code_health.collect_reach | ( | edges | ) |
Transitive reach, in both directions. This is where a god header shows up.
Cycle counts cannot express the damage darktable.h does, and this was measured
before the section was written: darktable.h reaches only 14 files downstream, so
at most 14 could ever cycle with it and exactly 3 do. Its 4-file cluster is
correct and says almost nothing.
The number that matters is the other direction. 552 of 741 files reach it, so
three quarters of the codebase depends on that one header, transitively. Editing
it rebuilds and re-reviews nearly everything, and no cycle metric will say so.
dependents how many files end up depending on this header, directly or not.
High means expensive to change and hard to reason about.
depth how many headers a translation unit drags in transitively. High
means slow builds and a file whose real interface is unknowable
from its own include list.
Definition at line 289 of file code_health.py.
Referenced by main().
| code_health.collect_selfcontained | ( | path | ) |
Fold in a header self-containment report, when one has been produced. A header should compile on its own. One that does not is relying on its includer having pulled something in first, which is the same defect as an unnecessary include seen from the other side: the dependency is real but written nowhere. Testing it needs a compiler and the project's include flags, so it is produced by the code-health workflow, which already configures a build tree, and consumed here.
Definition at line 522 of file code_health.py.
Referenced by main().
| code_health.collect_symbols | ( | db_path | ) |
Symbols per file, from Doxygen's SQLite output. memberdef.kind is Doxygen's own vocabulary: 'function', 'variable', 'typedef', 'macro definition', 'enumeration'. Note it is 'macro definition', not 'define'.
Definition at line 142 of file code_health.py.
References is_excluded(), and L.
Referenced by main().
| code_health.compute_stability | ( | mod_edges | ) |
Robert Martin's instability metric, and the violations it implies.
A second graph-derived view of the same question, independent of the ordering
above. Two independent derivations agreeing is worth more than either alone.
It is computed over directories, so it inherits their weakness as a unit - they
are drawers, not modules - and is reported for what it is:
Ca (afferent) how many modules depend on this one
Ce (efferent) how many modules this one depends on
I = Ce / (Ca + Ce) instability, 0 .. 1
I = 0 is a module everyone depends on and that depends on nothing: maximally
stable, expensive to change, and it had better be a leaf library. I = 1 is a
module nobody depends on: free to change, and it had better be a leaf consumer.
The Stable Dependencies Principle says a module should only depend on modules
at least as stable as itself. An edge A -> B with I(A) < I(B) breaks it: the
harder-to-change module was made to depend on the easier-to-change one, so the
volatile module's churn propagates into the stable one. That is the same defect
"layer inversion" is looking for, established without anyone declaring a layer.
Note the two can legitimately disagree, and where they do is interesting rather
than wrong: a widely used module that itself reaches into a volatile one scores
badly here even if the declared layers approve of it.
Definition at line 889 of file code_health.py.
Referenced by collect_layering().
| code_health.derive_layering | ( | edges, | |
| label | |||
| ) |
Summarise a feedback-arc-set ordering of one dependency graph.
Definition at line 869 of file code_health.py.
References feedback_arc_order(), L, and max.
Referenced by collect_layering().
| code_health.excluded_globs | ( | ) |
Shell-glob form of the exclusion list, for tools that filter by pattern. Derived on demand, never written out twice, so it always reflects the submodule paths loaded from .gitmodules.
Definition at line 87 of file code_health.py.
References L.
Referenced by collect_ccn().
| code_health.feedback_arc_order | ( | edges | ) |
Order nodes so that as few weighted edges as possible point backwards. `edges` maps (a, b) -> weight, meaning "a depends on b". Returns (order, back), where order[0] is the foundation and `back` lists the edges still pointing the wrong way: the minimum set of dependencies that would have to go for a layering to exist at all - a minimum feedback arc set. Nothing is declared. Topologically sorting instead would measure nothing (a topological order has no backward edges by construction) and does not exist anyway once the graph has a cycle, which is the interesting case. Eades-Lin-Smyth greedy: strip sinks to the back and sources to the front, and when neither exists - exactly when a cycle is in the way - remove the node with the largest outgoing-minus-incoming weight. Linear time, and at most |E|/2 - |V|/6 backward edges. Implemented with worklists rather than rescans so it stays linear on the file graph, which is two orders of magnitude larger than the directory graph.
Definition at line 781 of file code_health.py.
Referenced by collect_layering(), and derive_layering().
| code_health.include_edges | ( | db_path | ) |
Every (including file, included file) pair inside this tree. Doxygen's `includes` table is the same data its "included by" graphs are drawn from, so the numbers derived here and the graphs on the file pages cannot drift apart.
Definition at line 188 of file code_health.py.
References is_excluded(), and L.
Referenced by main().
| code_health.is_excluded | ( | path | ) |
True for anything that must not be measured: vendored, dead, or not shipped.
Definition at line 121 of file code_health.py.
References is_production_file(), and L.
Referenced by collect_ccn(), collect_clang_tidy(), collect_cloc(), collect_cppcheck(), collect_docs(), collect_git(), collect_symbols(), and include_edges().
| code_health.is_production_file | ( | path | ) |
True for a file that is compiled into the shipped application.
Definition at line 116 of file code_health.py.
References L.
Referenced by is_excluded().
| code_health.load_submodule_exclusions | ( | repo_root = "." | ) |
Extend EXCLUDED_DIR_PARTS with every path declared in .gitmodules.
Definition at line 62 of file code_health.py.
References L.
Referenced by main().
| code_health.main | ( | void | ) |
Definition at line 1953 of file code_health.py.
References build_markdown(), collect_ccn(), collect_clang_tidy(), collect_cloc(), collect_cppcheck(), collect_docs(), collect_git(), collect_god_header(), collect_includers(), collect_layering(), collect_modularity(), collect_reach(), collect_selfcontained(), collect_symbols(), include_edges(), L, load_submodule_exclusions(), main(), max, and source_include_edges().
Referenced by main().
| code_health.md_table | ( | headers, | |
| rows, | |||
aligns = None |
|||
| ) |
| code_health.module_of | ( | path, | |
source_dir = "src" |
|||
| ) |
The module a file belongs to: its first path component under the source dir.
Definition at line 724 of file code_health.py.
References L.
Referenced by collect_layering(), and collect_modularity().
| code_health.run | ( | cmd, | |
| ** | kw | ||
| ) |
Run a command, returning (ok, stdout). Never raises on a non-zero exit.
Definition at line 129 of file code_health.py.
References L.
Referenced by collect_ccn(), collect_cloc(), and collect_git().
| code_health.source_include_edges | ( | repo_root, | |
source_dir = "src" |
|||
| ) |
Build the include graph a second time, straight from the source text. Doxygen only records an include it managed to RESOLVE, and resolution depends on INCLUDE_PATH, on conditional compilation, and on which headers exist at doc-build time. Measured on darktable 5.6 it missed 184 edges that plainly exist in the files - platform headers behind #ifdef, mostly - while finding 133 the text does not show, from generated headers. Each graph therefore contained a cyclic cluster the other did not. Neither is authoritative on its own, so the panel unions them. Resolution mirrors the compiler: the including file's own directory first, then the source root.
Definition at line 227 of file code_health.py.
References L.
Referenced by main().
| code_health.strongly_connected | ( | nodes, | |
| succ | |||
| ) |
Tarjan's SCC, iterative so a deep include chain cannot blow the stack.
Definition at line 738 of file code_health.py.
Referenced by collect_layering().
| code_health.CLANG_TIDY_LINE |
Definition at line 1196 of file code_health.py.
| list code_health.EXCLUDED_DIR_PARTS |
Definition at line 48 of file code_health.py.
| code_health.PRODUCTION_LANGUAGES |
Definition at line 111 of file code_health.py.
| tuple code_health.SOURCE_SUFFIXES = (".c", ".cc", ".cpp", ".cxx", ".h", ".hpp", ".hxx", ".m", ".mm") |
Definition at line 106 of file code_health.py.
1.9.8