Skip to content

Use a short SPDX license header for LLM-centered files #1489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Dev-iL wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Use a short SPDX license header for LLM-centered files #1489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
857405f
Use a short SPDX license header where applicable; standard lic elsewhere
Dev-iL Feb 22, 2026
8fa6989
Specify hook stages explicitly
Dev-iL Mar 6, 2026
8be2028
Fix copyright notices
Dev-iL Mar 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .asf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,20 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features

github:
Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/core/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-core
description: Core Hamilton patterns for creating DAGs, applying decorators, testing, and debugging dataflows. Use for basic Hamilton development tasks.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(pytest:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->
Comment on lines 7 to +8
Copy link
Collaborator Author

@Dev-iL Dev-iL Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk I configured the hooks to accept the license being below the frontmatter using:

# .pre-commit-config.yaml
      - id: insert-license
          ...
          - --detect-license-in-X-top-lines
          - '8'

In case this comes in handy in Airflow too...


# Hamilton Core Development Assistant

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/hamilton-dev-workflow/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-dev-workflow
description: Systematic 5-step workflow for building Hamilton DAGs - DOT graphs, signatures, validation, TDD implementation. Use this workflow when creating new Hamilton modules from scratch.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(hamilton:*), Bash(pytest:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton Development Workflow for Claude Code

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/integrations/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-integrations
description: Hamilton integration patterns for Airflow, Dagster, FastAPI, Streamlit, Jupyter notebooks, and other frameworks. Use when integrating Hamilton with other tools.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(jupyter:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton Integrations

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/llm/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-llm
description: LLM and AI workflow patterns for Hamilton including RAG pipelines, embeddings, vector databases, and prompt engineering. Use for building AI applications with Hamilton.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(pytest:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton for LLM & AI Workflows

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/mcp/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-mcp
description: Interactive Hamilton DAG development via MCP tools. Validate, visualize, scaffold, and execute Hamilton pipelines without leaving the conversation. Use when building or debugging Hamilton dataflows interactively.
allowed-tools: Read, Grep, Glob, Bash(hamilton-mcp:*), Bash(python:*), Bash(pip:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton MCP Server -- Interactive DAG Development

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/observability/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-observability
description: Hamilton UI and SDK patterns for tracking, monitoring, and debugging dataflows. Use for observability, lineage tracking, and production monitoring.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(hamilton:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton Observability & UI

Expand Down
20 changes: 1 addition & 19 deletions .claude-plugin/skills/scale/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

---
name: hamilton-scale
description: Performance and parallelization patterns for Hamilton including async I/O, Spark, Ray, Dask, caching, and multithreading. Use for scaling Hamilton workflows.
allowed-tools: Read, Grep, Glob, Bash(python:*), Bash(pytest:*)
user-invocable: true
disable-model-invocation: false
---
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Hamilton Scaling & Performance

Expand Down
8 changes: 8 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SPDX-License-Identifier: Apache-2.0

# Files below use a short SPDX license identifier instead of the full
# Apache 2.0 header because they are NOT included in official releases
# (sdist / wheel). The `export-ignore` attribute ensures `git archive`
# (and tools that rely on it) will exclude them automatically.

.github export-ignore
2 changes: 2 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
<!-- SPDX-License-Identifier: Apache-2.0 -->

--- PR TEMPLATE INSTRUCTIONS (1) ---

Looking to submit a Apache Hamilton Dataflow to the sf-hamilton-contrib module? If so go the the `Preview` tab and select the appropriate sub-template:
Expand Down
2 changes: 2 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE/HAMILTON_CONTRIB_PR_TEMPLATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
<!-- SPDX-License-Identifier: Apache-2.0 -->

[Summary of contribution]

## For new dataflows:
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/hamilton-lsp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ jobs:
working-directory: dev_tools/language_server
steps:
- uses: actions/checkout@v3
- name: Check for missing Apache 2 license headers
run: python3 scripts/check_license_headers.py
working-directory: ${{ github.workspace }}
Comment on lines -26 to -28
Copy link
Contributor

@skrawcz skrawcz Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Collaborator Author

@Dev-iL Dev-iL Feb 24, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This answers all similar comments:

After the proposed change, license headers are being enforced by pre-commit hooks. This approach has several benefits:

  1. Contributors can tell there's an issue before getting to ci
  2. Coverage isn't lost since hooks should run on ci anyway as part of static checks
  3. No need to maintain license enforcement scripts
  4. Hooks were more thorough and detected missing licenses that the ci missed

Whereas the main downside is it's somewhat harder to customize if a specific file requires special treatment.

Copy link
Contributor

@skrawcz skrawcz Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue is people submit without running pre-commit hooks :(

Copy link
Collaborator Author

@Dev-iL Dev-iL Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assumption was that the hooks run on CI too. It's just that instead of a custom script we use a hook - which is more standard.

Copy link
Contributor

@skrawcz skrawcz Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add that to CI then?

Copy link
Collaborator Author

@Dev-iL Dev-iL Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It already runs as part of the "Unit Tests" job's "check linting with prek" stage. I'll move this to a separate "Static checks" job for better visibility.

Copy link
Collaborator Author

@Dev-iL Dev-iL Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
Expand Down
35 changes: 22 additions & 13 deletions .github/workflows/hamilton-main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,28 @@ on:
- 'pyproject.toml'

jobs:
static-checks:
name: "Static Checks"
runs-on: ubuntu-latest
env:
UV_PRERELEASE: "allow"
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install uv and set the python version
uses: astral-sh/setup-uv@v6
with:
python-version: '3.10'
enable-cache: true
cache-dependency-glob: "uv.lock"
activate-environment: true

- name: Check linting with prek
run: |
uv sync --dev
uv run prek run --all-files --hook-stage pre-merge-commit

test:
name: "Unit Tests"
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -59,19 +81,6 @@ jobs:
cache-dependency-glob: "uv.lock"
activate-environment: true

# It's enough to do it on single OS
- name: Check linting with prek
if: ${{ runner.os == 'Linux' }}
run: |
uv sync --dev
uv run prek install
uv run prek run --all-files

- name: Check for missing Apache 2 license headers
if: ${{ runner.os == 'Linux' }}
run: |
python3 scripts/check_license_headers.py

- name: Test hamilton main package
run: |
uv sync --group test
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/hamilton-sdk.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ jobs:
working-directory: ui/sdk
steps:
- uses: actions/checkout@v3
- name: Check for missing Apache 2 license headers
run: python3 scripts/check_license_headers.py
working-directory: ${{ github.workspace }}
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/hamilton-ui-backend.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,6 @@ jobs:
--health-retries 10
steps:
- uses: actions/checkout@v3
- name: Check for missing Apache 2 license headers
run: |
python3 scripts/check_license_headers.py
- name: Set up Python
uses: actions/setup-python@v4
with:
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/hamilton-ui-frontend.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@ jobs:
node-version: [20.x]
steps:
- uses: actions/checkout@v3
- name: Check for missing Apache 2 license headers
run: python3 scripts/check_license_headers.py
working-directory: ${{ github.workspace }}
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/sphinx-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,6 @@ jobs:
run: |
python -m pip install --upgrade --no-cache-dir pip setuptools

- name: Check for missing Apache 2 license headers
run: |
python3 scripts/check_license_headers.py

- name: Install Sphinx and dependencies
run: |
python -m pip install --upgrade --no-cache-dir sphinx sphinx-rtd-theme sphinx-simplepdf
Expand Down
Loading
Loading