Fix issues indexer document mapping#25619
Merged
silverwind merged 7 commits intogo-gitea:mainfrom Jul 4, 2023
wolfogre:bugfix/indexer_data
Merged
Fix issues indexer document mapping#25619silverwind merged 7 commits intogo-gitea:mainfrom wolfogre:bugfix/indexer_data
silverwind merged 7 commits intogo-gitea:mainfrom
wolfogre:bugfix/indexer_data
Conversation
Member
|
Does this migrate the existing index or does it reindex everything? |
lunny
approved these changes
Jul 1, 2023
Member
Author
It will reindex everything. We have no way to migrate indexes, either bleve or es. |
denyskon
approved these changes
Jul 4, 2023
Collaborator
|
I was unable to create a backport for 1.20. @wolfogre, please send one manually. 🍵 |
zjjhot
added a commit
to zjjhot/gitea
that referenced
this pull request
Jul 5, 2023
* giteaofficial/main: (22 commits)
[skip ci] Updated translations via Crowdin
Replace `interface{}` with `any` (go-gitea#25686)
Several fixes for mobile UI (go-gitea#25634)
Add elapsed time on debug for slow git commands (go-gitea#25642)
some less naked returns (go-gitea#25682)
Prevent duplicate image loading (go-gitea#25675)
Add unit test for `HashAvatar` (go-gitea#25662)
Fix the nil pointer when assigning issues to projects (go-gitea#25665)
Actions list enhancements (go-gitea#25601)
Fix issues indexer document mapping (go-gitea#25619)
Fix show more for image on diff page (go-gitea#25672)
Prevent SVG shrinking (go-gitea#25652)
Log the real reason when authentication fails (but don't show the user) (go-gitea#25414)
Add unit test for repository collaboration (go-gitea#25640)
Fix UI misalignment on user setting page (go-gitea#25629)
[skip ci] Updated translations via Crowdin
Correct translation wrong format (go-gitea#25643)
Add direct serving of package content (go-gitea#25543)
Fix bug when change user name (go-gitea#25637)
Make "cancel" buttons have proper type in modal forms (go-gitea#25618)
...
Member
Author
Since this PR is based on #25174 which is a refactor and hasn't been backported to v1.20, so it's difficult to backport this one. So I think it's OK to leave the old code in v1.20. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix regression of #5363 (so long ago).
The old code definded a document mapping for
issueIndexerDocType, and assigned it toBleveIndexerDataas its type. (BleveIndexerDatahas been renamed toIndexerDatain #25174, but nothing more.) But the old code never usedBleveIndexerData, it wrote the index with an anonymous struct type. Nonetheless, bleve would use the default auto-mapping for struct it didn't know, so the indexer still worked. This means the custom document mapping was always dead code.The custom document mapping is not useless, it can reduce index storage, this PR brings it back and disable default mapping to prevent it from happening again. Since
IndexerData(BleveIndexerData) has JSON tags, and bleve uses them first, so we should userepo_idas the field name instead ofRepoID.I did a test to compare the storage size before and after this, with about 3k real comments that were migrated from some public repos.
Before:
After:
It saves about half the storage space.