Repository navigation
The root of the problem is the root: How to get a root at all times in a codebase (or no root at all) #25
Description
The root problem is this:
- ScanCode TK demands a single root named Resource in a Codebase
- ScanCode TK creates a fake "virtual_root" name if there is no common root in several scans (--from-json /VirtualCodebases)
- ScanCode.io does not have a single root for a project (or rather it has one which might be the "codebase" directory, but never reports it.
- on Windows there is no single root: each "drive" is its own root.
- on POSIX, there is a single root, but it has no name: this is name only "/" and it can have files and directories as children
- ScanCode TK and CommonCode generally ignore or even strip a leading slash therefore making the POSIX root moot
- When you "--strip-root" in ScanCode TK (leaving aside the possible loss of data attached to a root dir) it is potentially problematic to further read this with --from-json because we have no root anymore.
- Scanning a subset of paths or a collections of path in ScanCode is problematic because of this need for root
- The Codebase classes expect some sort order when creating Resources which may not make sense in all cases and may be overly restrictive as we cannot predict this sort order at all times
- Somethings are not entirely clear:
- what is the different between a Codebase and a root Resource?
- why could a Codebase not be just a collection of paths? And why do we even need a root?
We need to define a clean and well spec way to handle this across all projects
Activity
[-]The root of the problem is the root: How to get a root at all times in resource[/-][+]The root of the problem is the root: How to get a root at all times in a codebase (or no root at all)[/+]pombredanne commentedon Jun 11, 2021
@JonoYang @tdruez @Pratikrocks @chinyeungli FYI
pratik0316 commentedon Jun 11, 2021
Generally when we are having a single root the
ridof 0 is assigned to the first resource and we are treating it as a rootpombredanne commentedon Jun 11, 2021
@Pratikrocks I think we likely do not need a root at all
pratik0316 commentedon Jun 11, 2021
@pombredanne I believe we are leveraging to some extent upon the
rootwhen we are walking the codebase then basically it is therootwe visit first .Merge pull request aboutcode-org#25 from nexB/new-configure
JonoYang commentedon Feb 24, 2023
As stated above, we need to align commoncode's Codebase/VirtualCodebase class and Resource class with scancode.io's Project and CodebaseResource model. I would like this done such that code written to work with commoncode's Codebase and Resource classes can work on scancode.io's Project and CodebaseResource model. This is important in the case of fingerprinting Package Resources using the matchcode-toolkit fingerprinting functions (https://github.com/nexB/purldb/blob/main/matchcode-toolkit/src/matchcode_toolkit/fingerprinting.py#L69) because the fingerprints depend on the order of Resources when we walk a codebase. We want the codebase walk to be the same across both classes.
One of the issues is that we need to reconcile is that commoncode creates a root resource for a Codebase where scancode.io does not create one for a Project. The root is used to determine where the root of the codebase is and helps us perform depth-first traversal into the Codebase.
An idea would be to treat the Codebase/VirtualCodebase class as the root resource, since the Codebase represents the entirety of the codebase being scanned. This would involve adding the Resource class fields to the Codebase/VirtualCodebase class, so we can return the Codebase like a Resource when we are walking a Codebase or Resource. Resources that are in the root of the project would have the Codebase as the parent Resource.
Some issues: