Skip to content

fitbit data sync optimization#690

Merged
CodeWithCJ merged 2 commits intomainfrom
dev
Feb 20, 2026
Merged

fitbit data sync optimization#690
CodeWithCJ merged 2 commits intomainfrom
dev

Conversation

@CodeWithCJ
Copy link
Owner

@CodeWithCJ CodeWithCJ commented Feb 20, 2026

Summary
This PR addresses several data accuracy issues and adds support for new health metrics in the Fitbit integration. The primary goal was to ensure that step data is reliable even when specific API summaries are missing and to fix a logic bug that caused activities to "shift" calendar days.

Key Changes
🛠 Bug Fixes & Accuracy
Date-Shifting Fix: Modified
processFitbitActivities
to extract the entry date directly from the local startTime string. This prevents late-night activities (e.g., an 8 PM walk) from being shifted to the next day due to UTC conversion.
Step Fallback Logic: Implemented a mechanism that sums steps from all individual activities (Walks, Runs, etc.) for a given day. If the official Fitbit daily summary is missing or reports a lower number, the system automatically uses the activity sum to update the daily total.
Respiratory Rate Mapping: Fixed a bug where respiratory rate data was missed; the processor now correctly checks both the standard and fullSleepSummary paths in the Fitbit response.
✨ New Features & Improvements
Exercise Diary Filtering: Activities named "Walk" are now filtered out from being created as separate exercise cards. This prevents diary clutter while ensuring their steps still contribute to the daily goal via the fallback logic.
New Metrics Support: Added synchronization for:
Cardio Fitness Score (VO2 Max)
Core Temperature
Duplicate Prevention: Integrated source_id checks (using Fitbit's logId) to prevent duplicate exercise entries during repeated syncs.
📝 Technical Improvements
Refined the use of
upsertStepData
to ensure the "best available truth" (official total vs. activity sum) is persisted in the check_in_measurements table.
Improved debug logging across the data processor for easier troubleshooting of sync discrepancies.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @CodeWithCJ, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the Fitbit data synchronization capabilities by integrating new data types such as Cardio Fitness (VO2 Max) and Core Temperature. It also optimizes existing activity processing, ensuring more accurate and comprehensive recording of user steps and exercise data, while improving the robustness of exercise entry management within the system. The changes aim to provide users with a richer and more reliable fitness tracking experience.

Highlights

  • Expanded Fitbit Data Integration: Added support for syncing Cardio Fitness (VO2 Max) and Core Temperature data from Fitbit, providing a more comprehensive health overview.
  • Enhanced Activity Processing: Implemented a fallback mechanism to update daily steps from individual activities and introduced logic to prevent redundant 'Walk' exercise entries when steps are already recorded.
  • Improved Exercise Entry Management: Refined the update and creation logic for exercise entries, including better handling of exercise snapshots, sets, and optimized duplicate checks for various data sources like HealthKit/HealthConnect.
  • Codebase Standardization: Standardized string literal usage from single to double quotes across several core files for improved consistency and maintainability.
Changelog
  • SparkyFitnessServer/integrations/fitbit/fitbitDataProcessor.js
    • Updated string literals from single to double quotes.
    • Added processFitbitCardioFitness and processFitbitCoreTemperature functions.
    • Modified processFitbitRespiratoryRate to check for breathing rate in multiple JSON paths.
    • Enhanced processFitbitActivities with step fallback logic and filtering for 'Walk' activities.
    • Exported new data processing functions.
  • SparkyFitnessServer/integrations/fitbit/fitbitService.js
    • Updated string literals from single to double quotes.
    • Expanded Fitbit authorization scope to include cardio_fitness.
    • Added fetchCardioFitnessScore and fetchCoreTemperature API fetching functions.
    • Integrated fetching and processing of Cardio Fitness and Core Temperature data into syncFitbitData.
    • Updated module exports to include new fetch functions.
  • SparkyFitnessServer/models/exerciseEntry.js
    • Updated string literals from single to double quotes.
    • Refined _updateExerciseEntryWithClient for more robust data merging and JSON field handling.
    • Improved createExerciseEntry to handle existing entries and skip duplicate checks for HealthKit/HealthConnect sources.
    • Enhanced getExerciseEntryById to safely parse JSON fields like equipment, muscles, and instructions.
    • Updated updateExerciseEntry and deleteExerciseEntriesByEntrySourceAndDate for improved sets management and logging.
Activity
  • No specific activity (comments, reviews, progress updates) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces several improvements to the Fitbit data sync process, including support for new metrics like VO2 Max and Core Temperature, and a fallback mechanism for step data. However, there are a few concerns regarding the efficiency of database operations during the sync and a potential bug where multiple Fitbit activities of the same type on the same day might overwrite each other. Additionally, the hardcoded exclusion of "Walk" activities should be reviewed as it may lead to data loss for users who track walks as exercises.

Comment on lines +332 to +334
const skipDuplicateCheck = ["HealthKit", "HealthConnect"].includes(
entrySource,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current logic for skipDuplicateCheck does not include Fitbit. This means that if a user has multiple activities of the same type (e.g., two separate runs) on the same day, the second one will overwrite the first one because the uniqueness check at line 347 only considers exercise_id and entry_date. Since Fitbit activities now include a unique source_id (the logId), Fitbit should be added to this list to allow multiple entries, or the uniqueness check should be updated to use source_id for better deduplication.

Suggested change
const skipDuplicateCheck = ["HealthKit", "HealthConnect"].includes(
entrySource,
);
const skipDuplicateCheck = ["HealthKit", "HealthConnect", "Fitbit"].includes(
entrySource,
);

Comment on lines +538 to +539
const entryDate = activity.startTime.substring(0, 10);
const startIso = parseFitbitTime(activity.startTime, timezoneOffset);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using substring(0, 10) on the raw activity.startTime is less robust than using the normalized ISO string from the parseFitbitTime helper. Additionally, the startIso variable is currently defined but unused in the rest of the loop, making it dead code. Using it to derive entryDate solves both issues.

Suggested change
const entryDate = activity.startTime.substring(0, 10);
const startIso = parseFitbitTime(activity.startTime, timezoneOffset);
const startIso = parseFitbitTime(activity.startTime, timezoneOffset);
const entryDate = startIso.split("T")[0];

Comment on lines +559 to +565
if (exerciseName.toLowerCase() === "walk") {
log(
"info",
`[fitbitDataProcessor] Skipping exercise entry for "Walk" as it is treated as steps only. Day: ${entryDate}`,
);
continue;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Hardcoding the exclusion of "Walk" activities may lead to data loss for users who track walks as exercises (e.g., for GPS maps, pace, or heart rate details). While these are also counted as steps, the exercise entry itself provides valuable context and historical data. Consider making this behavior configurable or only skipping walks that don't meet a certain threshold of duration or intensity.

Comment on lines +631 to +632
await measurementRepository.getCheckInMeasurementsByDate(userId, date);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Performing a database query (getCheckInMeasurementsByDate) inside a loop over stepsPerDay is inefficient, especially during a multi-day sync. It is recommended to fetch all measurements for the relevant date range in a single query using measurementRepository.getCheckInMeasurementsByDateRange before the loop and then perform the comparison in-memory.

@CodeWithCJ CodeWithCJ merged commit 12d2f6f into main Feb 20, 2026
5 of 6 checks passed
Sim-sat pushed a commit to Sim-sat/SparkyFitness that referenced this pull request Feb 20, 2026
- migrated the whole /pages/diary directory to tanstack query
- fixed unnecessary use of window listeners from CodeWithCJ#690
- fixes bug preventing to upload images
- splitting components in seperate files
- decouple logic from ui by moving the functions to utils
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant