Skip to content
Open

Docs #59

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: Deploy docs

on:
push:
branches:
- main

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: false

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.13'

- name: Install docs dependencies
run: pip install ".[docs]"

- name: Build docs
run: mkdocs build

- name: Upload pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: site/

deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ jobs:
run: make test

- name: Run example
run: resqui -c configurations/basic.json -t ${{ secrets.GITHUB_TOKEN }}
run: venv/bin/resqui -c configurations/basic.json -t ${{ secrets.GITHUB_TOKEN }}
42 changes: 27 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,25 +1,37 @@
PKGNAME=resqui
VENV=venv
PYTHON=$(VENV)/bin/python
PIP=$(VENV)/bin/pip

install:
pip install .
$(VENV)/bin/activate:
python3 -m venv $(VENV)
$(PIP) install -e ".[dev,docs]"

install-dev:
pip install -e ".[dev]"
install: $(VENV)/bin/activate

venv:
python3 -m venv venv
install-dev: $(VENV)/bin/activate

test:
python3 run_tests.py
install-docs: $(VENV)/bin/activate

example:
resqui -c configurations/basic.json
venv: $(VENV)/bin/activate

black:
black src/$(PKGNAME)
black tests
test: $(VENV)/bin/activate
$(PYTHON) run_tests.py

example: $(VENV)/bin/activate
$(VENV)/bin/resqui -c configurations/basic.json

black: $(VENV)/bin/activate
$(VENV)/bin/black src/$(PKGNAME)
$(VENV)/bin/black tests

docs: $(VENV)/bin/activate
$(VENV)/bin/mkdocs build

docs-serve: $(VENV)/bin/activate
$(VENV)/bin/mkdocs serve

clean:
rm -rf venv
rm -rf venv site

.PHONY: install install-dev venv test example black clean
.PHONY: install install-dev install-docs venv test example black docs docs-serve clean
63 changes: 63 additions & 0 deletions docs/explanation/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Architecture

## Overview

resqui is built around three concepts: **plugins**, **executors**, and a
**configuration-driven pipeline**. The CLI wires them together but the
components are independent.

```
CLI (cli.py)
├─ Configuration (config.py) ← JSON file or built-in defaults
├─ GitInspector (cli.py) ← extracts metadata from the repo
├─ IndicatorPlugin subclasses ← one per tool (HowFairIs, Gitleaks, …)
│ └─ uses Executor ← PythonExecutor or DockerExecutor
└─ Summary (core.py) ← aggregates CheckResults → JSON-LD
```

## Plugin system

Every quality check is a subclass of `IndicatorPlugin`. The CLI discovers
plugins at runtime via `__subclasses__()` — no registration step is needed.
Each plugin declares a list of indicator names in its `indicators` class
attribute; each name corresponds to a method of the same name on the class.

A single plugin instance is reused for all of its indicators within one run,
which means Docker images are pulled and Python venvs are created only once
per plugin class.

## Executor design

Plugins delegate subprocess execution to one of two executors:

**`PythonExecutor`** creates a temporary `venv` via the stdlib `venv` module,
installs the required packages with pip, and runs Python snippets inside it.
The venv is torn down in `__del__`. This approach keeps plugin dependencies
completely isolated from the resqui installation.

**`DockerExecutor`** pulls a Docker image on construction and runs commands
inside containers via `docker run`. The Docker daemon is the only external
dependency.

Both raise `ExecutorInitError` when they cannot initialise (Docker unavailable,
pip install failure, etc.). The CLI catches this and skips all indicators
belonging to that plugin with a warning, allowing the rest of the run to
continue.

## Why not extend IndicatorPlugin with Python magic?

Subclassing is used purely as a discovery mechanism — `__subclasses__()` gives
a flat list of all available plugins without any import-side-effect registration.
There is no `__init_subclass__` hook or metaclass. This keeps plugin authorship
simple: write a class, import it, and it is automatically available.

## Output format

`Summary.to_json()` serialises results as JSON-LD conforming to the EVERSE
Research Software Quality Assessment schema
(`https://w3id.org/everse/rsqa/0.0.1/`). Each `CheckResult` maps to one entry
in the `checks` array with linked indicator, software, and status IRIs.
65 changes: 65 additions & 0 deletions docs/explanation/indicators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Indicators

An **indicator** is a measurable property of a software repository. resqui maps
each indicator to a plugin method that performs the check automatically.

## Built-in indicators

### `has_license` — HowFairIs

Looks for a file named `LICENSE` or `LICENSE.md` in the repository root using
the [howfairis](https://github.com/fair-software/howfairis) library. Requires a
GitHub token.

W3ID: `https://w3id.org/everse/i/indicators/license`

### `has_citation` — CFFConvert

Checks for a valid `CITATION.cff` file using
[cffconvert](https://github.com/citation-file-format/cffconvert). Both
presence and schema validity are verified.

W3ID: `https://w3id.org/everse/i/indicators/citation`

### `has_ci_tests` — OpenSSFScorecard

Checks whether the project has a functioning CI test setup, as determined by
the [OpenSSF Scorecard](https://github.com/ossf/scorecard). Runs via Docker.
Requires a GitHub token.

### `human_code_review_requirement` — OpenSSFScorecard

Checks whether pull requests require human review before merging, per the
OpenSSF Scorecard "Code-Review" check.

### `has_published_package` — OpenSSFScorecard

Checks whether the project publishes a package to a registry such as PyPI or
npm, per the OpenSSF Scorecard "Packaging" check.

### `has_no_security_leak` — Gitleaks

Scans the repository history for accidentally committed secrets (API keys,
tokens, passwords) using [Gitleaks](https://github.com/gitleaks/gitleaks).
Runs via Docker.

## Interpreting results

Each indicator produces a `CheckResult` with:

| Field | Values |
|---|---|
| `output` | `valid` — indicator satisfied; `missing` — not found; `failed` — check error |
| `status` | Schema.org action status IRI |
| `evidence` | Human-readable finding from the underlying tool |

An indicator returning `missing` or `failed` does **not** abort the run — all
configured indicators are always attempted.

## Status IDs

| Status IRI | Meaning |
|---|---|
| `schema:CompletedActionStatus` | Check passed |
| `schema:FailedActionStatus` | Check ran but found a problem |
| `missing` | Check could not be completed (plugin skipped) |
103 changes: 103 additions & 0 deletions docs/how-to/add-a-plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Add an Indicator Plugin

This guide shows how to add a new quality indicator to resqui.

## 1. Create the plugin module

Add a new file under `src/resqui/plugins/`, for example `src/resqui/plugins/myplugin.py`.

## 2. Subclass `IndicatorPlugin`

```python
from resqui.plugins.base import IndicatorPlugin
from resqui.core import CheckResult


class MyPlugin(IndicatorPlugin):
name = "MyPlugin"
version = "1.0.0"
id = "https://w3id.org/everse/software/MyPlugin"
indicators = ["has_readme"]

def __init__(self, context):
# context.github_token and context.dashverse_token are available
# Raise PluginInitError here if required credentials are missing
pass

def has_readme(self, url, branch_or_commit):
# run your check here and return a CheckResult
found = ... # your check logic
return CheckResult(
process="Looks for a README file in the repository root.",
status_id="schema:CompletedActionStatus" if found else "schema:FailedActionStatus",
output="valid" if found else "missing",
evidence="Found README." if found else "No README found.",
success=found,
)
```

### Using PythonExecutor

```python
from resqui.executors.python import PythonExecutor

class MyPlugin(IndicatorPlugin):
def __init__(self, context):
self.executor = PythonExecutor(packages=["some-package==1.2.3"])

def my_indicator(self, url, branch_or_commit):
result = self.executor.execute(f"""
import some_package
output = some_package.check("{url}")
print(output)
""")
...
```

### Using DockerExecutor

```python
from resqui.executors.docker import DockerExecutor

class MyPlugin(IndicatorPlugin):
def __init__(self, context):
self.executor = DockerExecutor("ghcr.io/org/image:latest")

def my_indicator(self, url, branch_or_commit):
result = self.executor.run(["check", url])
...
```

Both executors raise `ExecutorInitError` on startup failure (e.g. Docker not
available, pip install failed). resqui catches this and skips the plugin with a
warning rather than aborting the whole run.

## 3. Export from the plugins package

Add an import to `src/resqui/plugins/__init__.py`:

```python
from resqui.plugins.myplugin import MyPlugin
```

## 4. Reference it in a configuration file

```json
{
"indicators": [
{
"name": "has_readme",
"plugin": "MyPlugin",
"@id": "missing"
}
]
}
```

## 5. Verify

```bash
resqui indicators
```

Your plugin and its indicators should appear in the list.
Loading
Loading