Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
ca922ad
add a dockerfile that can build the empty DB container
MoralCode Sep 4, 2025
4631d90
add empty db to GHCR container builds
MoralCode Sep 4, 2025
2c9a5f1
Merge pull request #3259 from MoralCode/empty-db-build
sgoggins Sep 10, 2025
33debfd
refactor send_messages to remove a almost entirely duplicate code path
MoralCode Sep 15, 2025
83d0db1
refactor facade_phase to return the group itself so that celery can p…
MoralCode Sep 15, 2025
bc35b38
fix(cli): display timezone in output for GitHub key expiry (#3251)
skools-here Sep 9, 2025
60d3036
removed day from formatted time
skools-here Sep 11, 2025
2598699
removed extra lines
skools-here Sep 15, 2025
139d5bc
Align timestamp formatting with header layout
skools-here Sep 16, 2025
a329409
Merge pull request #3273 from MoralCode/feature/facade-wtf
sgoggins Sep 16, 2025
fefb48a
Merge pull request #3261 from skools-here/fix-display-timezone-cli
sgoggins Sep 16, 2025
c8f22bb
Merge pull request #3271 from MoralCode/collection_util-duplicate-code
sgoggins Sep 16, 2025
db8843d
commented out abused API endpoint
sgoggins Sep 23, 2025
5899e01
bump celery and a related dependency
MoralCode Sep 23, 2025
493dddd
updated
sgoggins Sep 23, 2025
bcd58f5
Revert "Merge pull request #3271 from MoralCode/collection_util-dupli…
MoralCode Sep 23, 2025
4131e03
Merge pull request #3291 from chaoss/dependencies/celery-update
sgoggins Sep 23, 2025
84a4b8e
Merge pull request #3289 from chaoss/api-abuse-patch
sgoggins Sep 23, 2025
1e92217
Merge pull request #3292 from chaoss/regressions/revert-refactor
sgoggins Sep 23, 2025
b0bb3b8
updated metadata for new version
sgoggins Sep 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build_docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ jobs:
- database
- keyman
- rabbitmq
- empty_database
runs-on: ubuntu-latest
steps:
- name: Checkout repository
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Augur NEW Release v0.90.0
# Augur NEW Release v0.90.3

Augur is primarily a data engineering tool that makes it possible for data scientists to gather open source software community data - less data carpentry for everyone else!
The primary way of looking at Augur data is through [8Knot](https://github.com/oss-aspen/8knot), a public instance of 8Knot is available [here](https://metrix.chaoss.io) - this is tied to a public instance of [Augur](https://ai.chaoss.io).
Expand All @@ -11,7 +11,7 @@ We follow the [First Timers Only](https://www.firsttimersonly.com/) philosophy o
## NEW RELEASE ALERT!
**If you want to jump right in, the updated docker, docker-compose and bare metal installation instructions are available [here](docs/new-install.md)**.

Augur is now releasing a dramatically improved new version. It is also available [here](https://github.com/chaoss/augur/releases/tag/v0.90.0).
Augur is now releasing a dramatically improved new version. It is also available [here](https://github.com/chaoss/augur/releases/tag/v0.90.3).


- The `release` branch is a stable version of our new architecture, which features:
Expand Down
544 changes: 278 additions & 266 deletions augur/api/routes/contributor_reports.py

Large diffs are not rendered by default.

9 changes: 4 additions & 5 deletions augur/application/cli/github.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def update_api_key():
core_reset_header = "Core Reset Time"
graphql_request_header = "Graphql Requests Left"
graphql_reset_header = "Graphql Reset Time"
print(f"{'Key'.center(40)} {core_request_header} {core_reset_header} {graphql_request_header} {graphql_reset_header}")
print(f"{'Key'.center(40)} {core_request_header} {core_reset_header.center(24)} {graphql_request_header} {graphql_reset_header.center(24)}")
for key, core_key_data, graphql_key_data in valid_key_data:
core_requests = str(core_key_data['requests_remaining']).center(len(core_request_header))
core_reset_time = str(epoch_to_local_time_with_am_pm(core_key_data["reset_epoch"])).center(len(core_reset_header))
Expand Down Expand Up @@ -87,13 +87,12 @@ def update_api_key():

engine.dispose()


def epoch_to_local_time_with_am_pm(epoch):
local_time = datetime.fromtimestamp(epoch)
formatted_time = local_time.strftime('%I:%M %p') # This format includes the date as well
# Convert epoch to local time with timezone awareness
local_time = datetime.fromtimestamp(epoch).astimezone()
formatted_time = local_time.strftime('%I:%M %p %Z (UTC%z)').center(24)
return formatted_time


def find_duplicates(lst):
counter = Counter(lst)
return [item for item, count in counter.items() if count > 1]
Expand Down
15 changes: 6 additions & 9 deletions augur/tasks/git/facade_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@
facade_helper.log_activity('Debug',f"Commits missing from repo {repo_id}: {len(missing_commits)}")


if not len(missing_commits) or repo_id is None:

Check warning on line 262 in augur/tasks/git/facade_tasks.py

View workflow job for this annotation

GitHub Actions / runner / pylint

[pylint] reported by reviewdog 🐶 C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) Raw Output: augur/tasks/git/facade_tasks.py:262:7: C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len)
#session.log_activity('Info','Type of missing_commits: %s' % type(missing_commits))
return

Expand Down Expand Up @@ -491,7 +491,6 @@
#force_analysis = session.force_analysis
run_facade_contributors = facade_helper.run_facade_contributors

facade_sequence = []
facade_core_collection = []

if not limited_run or (limited_run and pull_repos):
Expand All @@ -509,14 +508,12 @@


#These tasks need repos to be cloned by facade before they can work.
facade_sequence.append(
group(
chain(*facade_core_collection),
process_dependency_metrics.si(repo_git),
process_libyear_dependency_metrics.si(repo_git),
process_scc_value_metrics.si(repo_git)
)
facade_sequence = group(
chain(*facade_core_collection),
process_dependency_metrics.si(repo_git),
process_libyear_dependency_metrics.si(repo_git),
process_scc_value_metrics.si(repo_git)
)

logger.info(f"Facade sequence: {facade_sequence}")
return chain(*facade_sequence)
return facade_sequence
2 changes: 1 addition & 1 deletion docker/backend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ RUN go install github.com/ossf/scorecard/v5@v5.1.1 \
FROM python:3.11-slim-bullseye

LABEL maintainer="outdoors@acm.org"
LABEL version="0.90.0"
LABEL version="0.90.3"

ENV DEBIAN_FRONTEND=noninteractive
ENV PATH="/usr/bin/:/usr/local/bin:/usr/lib:${PATH}"
Expand Down
2 changes: 1 addition & 1 deletion docker/database/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
FROM postgres:16

LABEL maintainer="outdoors@acm.org"
LABEL version="0.90.0"
LABEL version="0.90.3"

ENV POSTGRES_DB="test"
ENV POSTGRES_USER="augur"
ENV POSTGRES_PASSWORD="augur"

Check warning on line 9 in docker/database/Dockerfile

View workflow job for this annotation

GitHub Actions / End-to-end test (Docker)

Sensitive data should not be used in the ARG or ENV commands

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "POSTGRES_PASSWORD") More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/

Check warning on line 9 in docker/database/Dockerfile

View workflow job for this annotation

GitHub Actions / Push image (database)

Sensitive data should not be used in the ARG or ENV commands

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "POSTGRES_PASSWORD") More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/

Check warning on line 9 in docker/database/Dockerfile

View workflow job for this annotation

GitHub Actions / End-to-end test (Docker)

Sensitive data should not be used in the ARG or ENV commands

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "POSTGRES_PASSWORD") More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/

EXPOSE 5432

Expand Down
66 changes: 66 additions & 0 deletions docker/empty_database/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
from postgres:16 AS builder

ENV DEBIAN_FRONTEND=noninteractive

# Install uv (https://docs.astral.sh/uv/guides/integration/docker/#installing-uv)
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
ENV UV_COMPILE_BYTECODE=1
# The uv package cache will be on a cache volume, so can't be linked
ENV UV_LINK_MODE=copy
# Assert that the lockfile (uv.lock) is up-to-date. Use `uv lock` to update it
# manually if this fails the container build.
ENV UV_LOCKED=1

WORKDIR /augur

COPY pyproject.toml .
COPY uv.lock .
COPY .python-version .

# Install augur's dependencies early to take advantage of build cache
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-install-project --no-dev

# Copy in the actual code
# The RUN line below ensure that permissions are set correctly.
# This is the equivalent of the following docker --chmod flags, but done in a way thats compatible with podman.
# This can be removed once https://github.com/containers/buildah/issues/6066 or relevant equivalent is fixed
# - u=rw,u+X: user can read and write all files/dirs and execute directories
# - go=r,go+X: group and others can read all files/dirs and execute directories
COPY README.md .
COPY LICENSE .
COPY alembic.ini .
COPY augur/ augur/
COPY metadata.py .
COPY scripts/ scripts/

RUN find augur -type d -exec chmod u=rwx,go=rx {} + && find augur -type f -exec chmod u=rw,go=r {} +

RUN find scripts -exec chmod u=rwx,go=rx {} +

# Install the main project
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-dev

# We aren't going to activate the virtualenv (manually or via uv run), so we
# need adjust the PATH
ENV PATH="/augur/.venv/bin:${PATH}"

ENV POSTGRES_DB="augur"
ENV POSTGRES_USER="augur"
ENV POSTGRES_PASSWORD="augur"
ENV AUGUR_DB="postgresql+psycopg2://augur:augur@localhost:5432/augur"
# ENV PGDATA="/var/lib/postgresql/data"

RUN set -e && \
gosu postgres initdb && \
gosu postgres pg_ctl -D "$PGDATA" -o "-c listen_addresses='localhost'" -w start && \
gosu postgres psql -c "CREATE USER ${POSTGRES_USER} WITH SUPERUSER PASSWORD '${POSTGRES_PASSWORD}';" && \
gosu postgres psql -c "CREATE DATABASE ${POSTGRES_DB} OWNER ${POSTGRES_USER};" && \
augur db create-schema && \
gosu postgres pg_ctl -D "$PGDATA" -m fast -w stop


FROM postgres:16

COPY --from=builder /var/lib/postgresql/data /var/lib/postgresql/data
2 changes: 1 addition & 1 deletion docker/keyman/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
FROM python:3.11.12-alpine

LABEL maintainer="outdoors@acm.org"
LABEL version="0.90.0"
LABEL version="0.90.3"

RUN pip install --no-cache-dir --upgrade pip

RUN pip install --no-cache-dir redis==4.3.3

ENV KEYMAN_DOCKER=1
ENV PYTHONPATH="${PYTHONPATH}:/augur"

Check warning on line 11 in docker/keyman/Dockerfile

View workflow job for this annotation

GitHub Actions / End-to-end test (Docker)

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$PYTHONPATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

Check warning on line 11 in docker/keyman/Dockerfile

View workflow job for this annotation

GitHub Actions / Push image (keyman)

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$PYTHONPATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/

Check warning on line 11 in docker/keyman/Dockerfile

View workflow job for this annotation

GitHub Actions / End-to-end test (Docker)

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$PYTHONPATH' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/
# Ensure Python output is sent straight to terminal (e.g. for Docker logs)
ENV PYTHONUMBUFFERED=1

Expand Down
4 changes: 2 additions & 2 deletions metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

__short_description__ = "Python 3 package for free/libre and open-source software community metrics, models & data collection"

__version__ = "0.90.0"
__release__ = "v0.90.0 (Trade Deadline)"
__version__ = "0.90.3"
__release__ = "v0.90.3 (Trade Deadline)"

__license__ = "MIT"
__copyright__ = "University of Missouri, University of Nebraska-Omaha, CHAOSS, Sean Goggins, Brian Warner & Augurlabs 2025, Red Hat Software"
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ dependencies = [
"bokeh==2.0.2",
"boto3==1.17.57",
"bs4==0.0.1",
"celery==5.2.7",
"click==8.0.3",
"celery~=5.5",
"click~=8.1",
"cloudpickle>=0.2.2",
"coloredlogs==15.0",
"dask>=2021.6.2",
Expand Down
26 changes: 13 additions & 13 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading