Feature Request: Avoid Repeating Headlines

[stephen-hill]

I would like to request that the algorithm used for generating the current day's headlines should look back over the n previous days of articles (on a per category basis) to avoid repeating the same headlines everyday.

In the "Linux & OSS" category, the top headline for the past few days has repeatedly been about KDE Plasma 6.5.

This also happen in the "Formula 1" category. After a race weekend when news is slow, it will generate very similar headlines each day.

Thank you

[giorgiobrullo]

Hey, we can implement this - but it's unclear how this would help.

Categories being "slow" is not something we can artificially fix, at most the LLM would be able to mark the headline as an Update: instead of a new story. This was actually implemented today :), so tomorrow we should see the effects, but again it's unlikely that it solves the issue of stories repeating.

What would you expect to happen in this situation?

[stephen-hill]

Thanks for adding marking the headline as updated, it'll be nice to see how to feature works.

Initially I had thought that if there is no new news, then the category should empty: "No new today". Which maybe makes sense for an existing user, but it doesn't for a new user.

Therefore, I think it should simply show the previous day(s) news articles. An existing user will have read them, so they will be blue ticks. A new user would just see unread stories. The UI would need updating to have an "updated x hours/days ago" on a per category basis, rather than the whole site.

I agree though, there isn't an easy solution to this, and it only affects niche categories with slow news.

Another solution might be (and I'm making assumptions about how the system works) for the LLM to first identify if a story is potentially repeated from previous days, and put that story to the bottom of the candidate list - or add a counter for how many previous days it has been mentioned. It should then continue searching for headlines. The candidate list of headlines should be bigger than the final list, and could be sorted by new headlines at the top, repeated headlines at the bottom. Hopefully repeated news articles at the bottom then get truncated.

Not an easy problem to solve, but thanks for your work :)

[ChewingBever]

I was going to report this issue myself, but figured I would piggyback off this issue instead.

I read Kagi News via RSS, and the constantly repeating headlines are rather annoying. Especially in slow categories like cybersecurity, it's basically the same articles being reported every day, with almost identical titles and contents.

Imo it would be better for each headline to only be reported on once, older articles can simply be kept in the UI (and the rss feed) longer, without any changed content. This would allow RSS readers to correctly filter out the duplicate entries.