Skip to content

Commit fde8bfb

Browse files
zachshallbetterzachshallbetter
committed
documentation and directory improvement. Readme modifications (again)
1 parent 6b211e9 commit fde8bfb

15 files changed

+706
-304
lines changed

‎.env.example

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Model settings
2+
MODEL_CACHE_DIR=.cache/models
3+
TOKENIZERS_PARALLELISM=true
4+
5+
# Training settings
6+
CUDA_VISIBLE_DEVICES=0,1
7+
TORCH_DISTRIBUTED_DEBUG=INFO
8+
9+
# Resource limits
10+
MAX_GPU_MEMORY=0.9
11+
MAX_CPU_THREADS=8
12+
13+
# Logging
14+
LOG_LEVEL=INFO
15+
TENSORBOARD_DIR=runs

‎.github/workflows/trunk-check.yml

+20-2
Original file line numberDiff line numberDiff line change
@@ -9,21 +9,39 @@ on:
99
jobs:
1010
trunk-check:
1111
runs-on: ubuntu-latest
12+
timeout-minutes: 10
1213

1314
steps:
14-
- uses: actions/checkout@v3
15+
- name: Checkout
16+
uses: actions/checkout@v3
17+
with:
18+
fetch-depth: 1
1519

1620
- name: Set up Python 3.11
1721
uses: actions/setup-python@v4
1822
with:
1923
python-version: "3.11"
20-
cache: 'pip'
24+
cache: "pip"
25+
26+
- name: Cache Trunk
27+
uses: actions/cache@v3
28+
with:
29+
path: ~/.cache/trunk
30+
key: trunk-${{ runner.os }}-${{ hashFiles('.trunk/trunk.yaml') }}
31+
restore-keys: |
32+
trunk-${{ runner.os }}-
2133
2234
- name: Install dependencies
2335
run: |
2436
python -m pip install --upgrade pip
2537
pip install -e ".[dev]"
2638
39+
- name: Trunk Check
40+
uses: trunk-io/trunk-action@v1
41+
with:
42+
check-mode: all
43+
arguments: --github-annotate notice
44+
2745
- name: Run pre-commit checks
2846
run: |
2947
pre-commit run --all-files

‎.gitignore

+28-38
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ MANIFEST
2626
venv/
2727
env/
2828
ENV/
29-
.env
3029
.venv
3130
.python-version
3231

@@ -38,39 +37,36 @@ ENV/
3837
.project
3938
.pydevproject
4039
.settings/
40+
.cursor
4141

42-
# Testing
42+
# Testing and Coverage
4343
.coverage
4444
coverage.xml
4545
htmlcov/
4646
.pytest_cache/
4747
.tox/
4848
nosetests.xml
49-
.coverage
50-
.logs
51-
49+
test-results/
50+
performance-results/
51+
distributed-test-results/
5252

53-
# Type Checking
54-
.type_check_cache/
53+
# Type Checking and Linting
5554
.mypy_cache/
55+
.ruff_cache/
5656
.pyre/
5757
.pytype/
58+
.trunk/out/
59+
.trunk/logs/
60+
.trunk/notifications/
61+
.trunk/plugins/
62+
.trunk/tools/
5863

59-
# Distribution
60-
*.pyc
61-
*.pyo
62-
*.pyd
63-
*.so
64-
*.dll
65-
*.dylib
66-
67-
# Project Specific
68-
poetry.lock
69-
.llamahome/
64+
# Project Data and Cache
7065
.data/
71-
*.log
7266
.cache/
67+
.logs/
7368
benchmark_results/
69+
.llamahome/
7470

7571
# Documentation
7672
docs/_build/
@@ -83,25 +79,19 @@ site/
8379
.DS_Store
8480
Thumbs.db
8581

86-
# Cache directories
87-
.cache/
88-
.pytest_cache/
89-
__pycache__/
90-
*.py[cod]
91-
92-
# Environment
93-
.env
94-
.venv/
95-
venv/
96-
ENV/
97-
98-
# Python package metadata
99-
*.egg-info/
100-
*.egg
82+
# Distribution
83+
*.pyc
84+
*.pyo
85+
*.pyd
86+
*.so
87+
*.dll
88+
*.dylib
10189
dist/
10290
build/
10391

104-
# IDE
105-
.idea/
106-
.vscode/
107-
.env.example
92+
# Keep Important Config Files
93+
!.trunk/trunk.yaml
94+
!.trunk/configs/
95+
!.github/workflows/
96+
!.env.example
97+

‎.trunk/README.md

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Trunk Configuration
2+
3+
This directory contains configuration files and tools for [Trunk](https://trunk.io), our unified code quality and linting tool.
4+
5+
## Directory Structure
6+
7+
```
8+
.trunk/
9+
├── configs/ # Language-specific linter configurations
10+
│ ├── .markdownlint.yaml # Markdown linting rules
11+
│ └── .yamllint.yaml # YAML linting rules
12+
├── out/ # Generated output files (gitignored)
13+
├── logs/ # Trunk operation logs (gitignored)
14+
├── notifications/ # Trunk notification data (gitignored)
15+
├── plugins/ # Trunk plugins (gitignored)
16+
├── tools/ # Cached tool installations (gitignored)
17+
└── trunk.yaml # Main Trunk configuration
18+
```
19+
20+
## Configuration
21+
22+
Our Trunk setup includes:
23+
24+
### Linters
25+
26+
- **Python**
27+
28+
- `black` - Code formatting
29+
- `isort` - Import sorting
30+
- `mypy` - Static type checking
31+
- `ruff` - Fast Python linter
32+
- `bandit` - Security linting
33+
34+
- **Documentation & Config**
35+
36+
- `markdownlint` - Markdown formatting
37+
- `yamllint` - YAML validation
38+
- `prettier` - General formatting
39+
40+
- **Security**
41+
- `trufflehog` - Secret scanning
42+
- `git-diff-check` - Whitespace error detection
43+
44+
### Actions
45+
46+
- `trunk-announce` - Notify about Trunk events
47+
- `trunk-check-pre-push` - Run checks before git push
48+
- `trunk-fmt-pre-commit` - Format code before commits
49+
- `trunk-upgrade-available` - Check for Trunk updates
50+
51+
## Usage
52+
53+
Common commands:
54+
55+
```bash
56+
# Check all files
57+
trunk check --all
58+
59+
# Format all files
60+
trunk fmt --all
61+
62+
# Check specific files
63+
trunk check path/to/file.py
64+
65+
# Run specific linter
66+
trunk check --all --filter=black
67+
```
68+
69+
## Integration
70+
71+
Trunk is integrated with our CI/CD pipeline through GitHub Actions:
72+
73+
- Pre-commit hooks for code formatting
74+
- Pre-push hooks for comprehensive checks
75+
- Automated PR checks with inline annotations
76+
- Caching for faster CI runs
77+
78+
## Cache Management
79+
80+
Trunk caches tools and configurations in:
81+
82+
- `.trunk/tools/` - Linter binaries
83+
- `.trunk/out/` - Linter outputs
84+
- `~/.cache/trunk` - Global cache (CI uses this)
85+
86+
All cache directories are gitignored.
87+
88+
## Updating
89+
90+
To update Trunk and its tools:
91+
92+
```bash
93+
trunk upgrade --all
94+
```
95+
96+
Configuration changes should be made in:
97+
98+
- `trunk.yaml` for tool versions and general settings
99+
- `.trunk/configs/` for language-specific rules

‎.trunk/trunk.yaml

+29-8
Original file line numberDiff line numberDiff line change
@@ -16,26 +16,47 @@ lint:
1616
- [email protected]
1717
- [email protected]
1818
- [email protected]
19+
- [email protected]
20+
- git-diff-check@SYSTEM
21+
- [email protected]
22+
- [email protected]
1923

2024
ignore:
2125
- linters: [ALL]
2226
paths:
2327
- "**/node_modules/**"
2428
- "**/.venv/**"
25-
- .cache/
26-
- .data/
27-
- .trunk/
28-
- .logs/
29-
- .github/
29+
- .cache/**
30+
- .data/**
31+
- .trunk/out/**
32+
- .trunk/logs/**
33+
- .logs/**
3034
- "*.pyc"
3135
- "*.pyo"
3236
- "*.pyd"
3337
- "*.so"
38+
- "**/__pycache__/**"
3439

35-
runtimes:
36-
enabled:
37-
- [email protected]
40+
- linters: [trufflehog]
41+
paths:
42+
- .env.example
43+
- docs/**/*.md
44+
- "**/test_*.py"
45+
46+
# Commenting out runtime as we're using system Python
47+
# runtimes:
48+
# enabled:
49+
# - [email protected]
3850

3951
actions:
4052
enabled:
53+
- trunk-announce
54+
- trunk-check-pre-push
55+
- trunk-fmt-pre-commit
4156
- trunk-upgrade-available
57+
58+
tools:
59+
enabled:
60+
- [email protected]
61+
- [email protected]
62+
- [email protected]

‎Makefile

+26
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,32 @@ else
5656
RM := rm -rf
5757
endif
5858

59+
# Data directories
60+
MODELS_DIR := $(DATA_DIR)/models
61+
DATASETS_DIR := $(DATA_DIR)/datasets
62+
EMBEDDINGS_DIR := $(DATA_DIR)/embeddings
63+
CHECKPOINTS_DIR := $(DATA_DIR)/checkpoints
64+
ARTIFACTS_DIR := $(DATA_DIR)/artifacts
65+
MEMORY_DIR := $(DATA_DIR)/memory
66+
METRICS_DIR := $(DATA_DIR)/metrics
67+
TELEMETRY_DIR := $(DATA_DIR)/telemetry
68+
TRAINING_DIR := $(DATA_DIR)/training
69+
LOCAL_DIR := $(DATA_DIR)/local
70+
71+
# Create data directories
72+
.PHONY: setup-data
73+
setup-data:
74+
mkdir -p $(MODELS_DIR) \
75+
$(DATASETS_DIR) \
76+
$(EMBEDDINGS_DIR) \
77+
$(CHECKPOINTS_DIR) \
78+
$(ARTIFACTS_DIR) \
79+
$(MEMORY_DIR) \
80+
$(METRICS_DIR) \
81+
$(TELEMETRY_DIR) \
82+
$(TRAINING_DIR) \
83+
$(LOCAL_DIR)
84+
5985
# Default target
6086
.PHONY: all
6187
all: setup

0 commit comments

Comments
 (0)