Skip to content

Comments

add test for translations#2679

Merged
hayescode merged 4 commits intoChainlit:mainfrom
codomposer:feat/test_translation
Nov 20, 2025
Merged

add test for translations#2679
hayescode merged 4 commits intoChainlit:mainfrom
codomposer:feat/test_translation

Conversation

@codomposer
Copy link
Contributor

@codomposer codomposer commented Nov 20, 2025

Add Comprehensive Tests for Translations Module

Overview

This PR adds comprehensive test coverage for the translations module, ensuring robust testing of JSON structure comparison and translation file linting functionality.

Changes

  • New Test File: backend/tests/test_translations.py with 41 test cases
  • Test Coverage: JSON structure comparison, translation linting, and edge cases
  • Validation: Input validation, error detection, and output formatting

Test Suites

1. JSON Structure Comparison Tests (26 tests)

Basic Comparisons (5 tests)

  • Identical Structures: Validates matching JSON structures return no errors
  • Missing Keys: Detects keys present in truth but absent in comparison
  • Extra Keys: Identifies keys in comparison not present in truth
  • Both Missing and Extra: Handles mixed scenarios with multiple error types
  • Empty Dictionaries: Validates empty dict comparisons

Nested Structure Tests (8 tests)

  • Nested Structures: Validates deeply nested JSON objects
  • Nested Missing Keys: Detects missing keys in nested structures with proper path notation
  • Nested Extra Keys: Identifies extra keys in nested structures
  • Deeply Nested: Tests structures with multiple nesting levels (a.b.c.d.e.f)
  • Complex Nested: Multiple levels with mixed error types
  • Path Formatting: Ensures error paths use correct dot notation without leading dots

Structure Mismatch Tests (2 tests)

  • Dict vs Value: Detects when truth has dict but comparison has value
  • Value vs Dict: Detects when truth has value but comparison has dict

Input Validation Tests (3 tests)

  • Non-dict Truth: Raises ValueError when truth is not a dictionary
  • Non-dict Comparison: Raises ValueError when to_compare is not a dictionary
  • Both Non-dict: Validates error when both inputs are invalid

Value Type Tests (4 tests)

  • Different Value Types: Confirms leaf node values are not compared (structure only)
  • Null Values: Handles None/null values correctly
  • List Values: Treats lists as leaf nodes (structure matches)
  • Empty String Values: Validates empty strings are handled correctly

Edge Cases (4 tests)

  • Empty Truth with Data: Detects all keys as extra when truth is empty
  • Empty Comparison with Data: Detects all keys as missing when comparison is empty
  • Complex Structure: Multi-level nested objects with various error types
  • Path Formatting: Validates error message path construction

2. Translation Linting Tests (6 tests)

Success Cases (1 test)

  • No Errors: Outputs success message with ✅ emoji when structures match

Error Detection (4 tests)

  • With Errors: Displays missing and extra key errors
  • Nested Errors: Shows errors with proper nested path notation
  • Structure Mismatch: Reports structure type mismatches
  • Multiple Errors: Handles and displays multiple error types simultaneously

Output Formatting (1 test)

  • Output Format: Validates proper formatting with newlines and messages

3. Edge Cases Tests (9 tests)

Special Key Types (3 tests)

  • Numeric Keys: Handles numeric strings as keys
  • Special Characters: Supports keys with -, _, . characters
  • Unicode Keys: Validates unicode characters in keys (Chinese, French, Arabic)

Nesting and Values (3 tests)

  • Very Deeply Nested: Tests extreme nesting levels (6+ levels deep)
  • Empty String Values: Handles empty strings as valid values
  • Error Ordering: Validates consistent error reporting order

Linting Edge Cases (3 tests)

  • Empty Filename: Handles empty string filenames gracefully
  • Error Preservation: Ensures errors are categorized correctly (extra vs missing)
  • Consistent Output: Validates output format consistency

Technical Implementation

Structure Comparison Algorithm

def compare_json_structures(truth, to_compare, path=""):
    # Recursive comparison of nested dictionaries
    # - Identifies extra keys: to_compare_keys - truth_keys
    # - Identifies missing keys: truth_keys - to_compare_keys
    # - Detects structure mismatches (dict vs non-dict)
    # - Builds error paths using dot notation

Error Types

  • ❌ Missing key: Key exists in truth but not in comparison
  • ⚠️ Extra key: Key exists in comparison but not in truth
  • ❌ Structure mismatch: Type mismatch (dict vs value)

Path Notation

  • Root level: 'key'
  • Nested: 'parent.child.grandchild'
  • No leading dots for root-level keys

Coverage Details

  • Total Tests: 41
  • Structure Comparison: 26 tests covering all comparison scenarios
  • Linting: 6 tests covering output and error reporting
  • Edge Cases: 9 tests covering special characters, unicode, and extreme cases
  • Input Validation: Proper error handling for invalid inputs

Testing Approach

  • Assertion-based: Direct comparison of error lists and output strings
  • Output Capture: Uses StringIO with patch to capture print statements
  • Error Validation: Checks for specific error messages and emojis
  • Path Validation: Ensures proper dot notation in nested error paths
  • Unicode Support: Tests international character support

Related Modules

  • chainlit.translations: JSON structure comparison and linting utilities
  • Translation files in frontend/translations/ directory
  • i18n configuration and language support

Contribution by Gittensor, learn more at https://gittensor.io/

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. backend Pertains to the Python backend. labels Nov 20, 2025
@hayescode hayescode enabled auto-merge November 20, 2025 15:58
# Add Comprehensive Tests for Cache Module

## Overview
This PR adds comprehensive test coverage for the `cache` module,
ensuring robust testing of the `@cache` decorator functionality and
LangChain cache initialization.

## Changes
- **New Test File**: `backend/tests/test_cache.py` with 27 test cases
- **Test Coverage**: Cache decorator behavior, thread safety, LangChain
integration, and edge cases
- **Module Access**: Uses `sys.modules` to access the internal `_cache`
dictionary for test isolation

## Test Suites

### 1. Cache Decorator Tests (14 tests)
- **Basic Functionality** (3 tests)
  - Caches function results correctly
  - Different arguments create separate cache entries
  - Returns cached results on subsequent calls with same arguments

- **Keyword Arguments** (3 tests)
  - Works with keyword arguments
  - Order-independent kwargs (sorted for cache key)
  - Mixed positional and keyword arguments

- **Special Cases** (4 tests)
  - Functions with no arguments
  - Mutable return values (returns same object reference)
  - `None` as argument value
  - Preserves original function behavior (including exceptions)

- **Advanced Features** (4 tests)
  - Thread-safe cache access with `_cache_lock`
  - Different functions with same args have separate cache entries
  - Function name included in cache key
  - Cache decorator doesn't interfere with function execution

### 2. LangChain Cache Initialization Tests (7 tests)
- **Configuration Handling**
  - Cache disabled by `config.project.cache = False`
  - Cache disabled by `config.run.no_cache = True`
  - No initialization when LangChain not installed

- **LangChain Integration**
  - Initializes `SQLiteCache` when LangChain is available
  - Uses configured `lc_cache_path` for database location
  - Logs info message when creating new cache file
  - Skips initialization when `lc_cache_path` is `None`

### 3. Edge Cases Tests (6 tests)
- **Argument Types**
  - Unhashable arguments (lists) raise `TypeError`
  - String arguments work correctly
  - Tuple arguments (hashable) work correctly
  - Boolean arguments work correctly

- **Cache State**
  - Global cache state persists across function calls
  - Cache entries accumulate for different arguments

## Technical Implementation

### Cache Key Generation
```python
cache_key = (
    (func.__name__,) + args + tuple((k, v) for k, v in sorted(kwargs.items()))
)
```
- Function name ensures different functions don't collide
- Sorted kwargs ensure order independence
- Tuple structure makes keys hashable

### Thread Safety
```python
with _cache_lock:
    if cache_key not in _cache:
        _cache[cache_key] = func(*args, **kwargs)
```
- Uses `threading.Lock()` to prevent race conditions
- Ensures only one thread computes result for a given key
- All threads get the same cached result

### Test Isolation
```python
cache_module = sys.modules["chainlit.cache"]

def setup_method(self):
    cache_module._cache.clear()
```
- Accesses actual module via `sys.modules` (not the re-exported
function)
- Clears cache before/after each test for isolation
- Prevents test interference

## Coverage Details
- **Total Tests**: 27
- **Cache Decorator**: 14 tests covering caching logic, thread safety,
and edge cases
- **LangChain Init**: 7 tests covering configuration and integration
- **Edge Cases**: 6 tests covering argument types and cache state
- **Mocking Strategy**: Uses `patch` for config, `patch.dict` for
`sys.modules`, `Mock` for LangChain components

## Testing Approach
- **Decorator Testing**: Uses `call_count` to verify functions are
called only when cache misses
- **Thread Safety**: Spawns multiple threads to verify single execution
- **LangChain Mocking**: Patches `importlib.util.find_spec` and mocks
LangChain modules
- **State Verification**: Checks `_cache` dictionary size and contents
- **Exception Handling**: Verifies decorator preserves original function
exceptions

## Related Modules
- `chainlit.cache`: Cache decorator and LangChain cache initialization
- `chainlit.config`: Configuration for cache settings
- `langchain.cache`: SQLiteCache for LangChain LLM caching (optional
dependency)
- `langchain.globals`: Global LLM cache setter (optional dependency)

Contribution by Gittensor, learn more at https://gittensor.io/
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

auto-merge was automatically disabled November 20, 2025 16:20

Head branch was pushed to by a user without write access

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Nov 20, 2025
@codomposer
Copy link
Contributor Author

@hayescode
could you check this pr again?
there was an error in cypress test in previous attempt, so I rebased main branch, and seems fine now

@codomposer codomposer requested a review from hayescode November 20, 2025 16:42
@hayescode hayescode added this pull request to the merge queue Nov 20, 2025
Merged via the queue into Chainlit:main with commit 1aa9ad1 Nov 20, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Pertains to the Python backend. size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants