Issues and Discussions for documentation

[TonyGravagno]

I don't think many people know about the spamd/rspamd.com repo which contains all of the site documentation.

Can we add more references around the code project and in the docs themselves, so that people know where to go to ask questions about doc content and note issues with the doc pages?

I'm also hoping we can create a Discussions section in the docs repo. If someone asks "how does this feature work?", that's a question for the rspamd code project. But if they say "I don't understand this text", that's a doc issue. It would be helpful to separate those components.

When someone asks a question in one of the Support groups, the answers are lost if not transferred into the documentation. The time taken to answer a question in one medium doesn't help those monitoring another. It would be better if the answer is always "Here is the answer, someone should create a ticket to get this into the docs". With a ticket created, perhaps some discussion to formulate good docs, and a reference to the original Q&A, we can then refer back from the doc for more complete information on a lot of topics.

If this initiative is approved, I'll take the time to add notes to the ReadMe pages, Support, and other places, and I'll monitor some of the discussion areas to encourage people to contribute doc notes and/or to submit requests for specific doc changes.

Thanks.

[fatalbanana]

Can we add more references around the code project and in the docs themselves, so that people know where to go to ask questions about doc content and note issues with the doc pages?

What (or where) would you propose in particular? There is an Edit Page button on most pages of the website; perhaps it could be made more visible.

I'm also hoping we can create a Discussions section in the docs repo.

Possibly Issues might be enough.

When someone asks a question in one of the Support groups, the answers are lost if not transferred into the documentation.

Commonly answers to questions are already documented though people may struggle to find the relevant documentation or not take time to properly read it. Sometimes, though it's rare, things which are conspicuously missing from docs may come up & that may unfortunately be forgotten about.

[TonyGravagno]

Thanks for the responses.

I think Edit Page can be intimidating, implying a commitment for someone to make a change themselves when they really just want to make a suggestion. For many, if they don't understand the documentation, they have nothing to contribute with an edit. The proposal here is to ensure that there is a clear path to asking questions, asking for clarity, and suggesting improvements.

As to Issues vs Discussions, you see the difference between the two in the code repo, and there are lively discussions in GH itself about the value of Discussions and how it can/should/will be used. Suffice to say, there is a difference between an issue and a discussion, and the Issues mechanism has not served repositories well when people are asking questions. I'm suggesting we use the tools available for their best effect.

I haven't noticed anything conspicuously missing from the docs - kudos and gratitude to the team. Yes, there is documentation that people don't read, everywhere on the planet. :) But when people read the docs and they still have an issue, that should be a clue that the docs may need clarity. I've had discussions with mail server administrators who don't understand many aspects of Rspamd. They're not lazy or stupid. They've read the docs and they still don't get some details because some of the material is difficult to read, not enough examples, outdated, etc. And while not a professional mail server administrator, I do have over 40 years experience with many environments, and I came here asking the same kind of questions that others have because the docs didn't provide the required clarity.

I'm not saying the docs are incomplete or that there aren't enough docs. On the contrary, this is one of the best documented platforms there is, in terms of volume of content. That says a lot given the complexity. I am saying that there are still a lot of questions that people have after reading the docs and I hope we can make it easier to provide a firm "RTM, your answer is right here". That can save you guys a lot of time in the public spaces.

For me, the issue seems to be that there is an underlying consistency in the entire platform that is missed in focused documentation. The material is reference material, written by and for people who already understand the platform. It's not tutorial material to help establish an overall understanding which then makes the reference material valuable to fill in the details. Until tutorials are available (easier to find?) I am suggesting that we do what we can to invite concerns about the existing docs so that they can be made more useful for all levels of users.

To be specific:

• When posting responses in the various chat areas, suggest that people not only visit the documentation, but also that they should create an Issue for the documentation where they feel more clarity would be helpful.
• Add a note to the platform Readme that the documentation is also open-source, and requests and contributions are welcome.
• The documentation itself should be clear that it is not intended as a tutorial but as reference, that we understand that it's a big package that requires a lot of understanding to connect the dots and make sense of the entire platform, and that we welcome suggestions as Issues to point to areas where the material requires other background detail or more examples.

As I said, I'll get involved with all of this because one of the best ways for me to learn is to teach, in the form of writing new material. If we do this and it doesn't help after a year or two, we can take it down a notch again.

Thanks again.

[vstakhov]

I would say that the main problem of the documentation is that it is a reference in fact with a few guides that could easily be lost in the amount of reference materials. Like all plugins, options, various Lua API references... Even FAQ section is quite bloated. The quick start guide is not bad but it is written by me, who authored Rspamd as well. So I imply that things are easy whilst they are not.

I have talked with Pieter Hollants who has written https://www.0xf8.org/2018/05/an-alternative-introduction-to-rspamd-configuration-introduction/
and he told me that our docs are mostly written from the perspective of developers, so they describe how things are done. They do not describe how to do things using Rspamd and what should be done to make Rspamd efficient. I have tried to write some guides (e.g. https://rspamd.com/doc/lua/examples.html) but they just got lost in all those references...

Anyway, I think you have a similar impression about the Rspamd documentation overall.

[TonyGravagno]

We're 100% on the same page. I was just looking at the organization of the material, and noted, for example, that details for writing rules are first in th Developers documentation, while the Architecture follows that. Similarly, many of the pages require an understanding from some other pages. That's typical of the entire doc site. I completely understand this. I have my own commercial products, write my own docs, and we fill in the material as best we can whenever we have time.

At the moment I'm not suggesting an overhaul. What's here is great. It just needs some structure for people to follow end-to-end, as best as we can guide that process. Rather than writing a new set of docs for that purpose, I'm suggesting more basic info mixed in with the detail to provide context, more cross-linking of concepts, and over time more pages that rise a bit over the trees so that we can see some of the forest.

I don't have a ton of time for this either, so I do not propose any huge changes that I personally couldn't support. My motivation is that I want to use Rspamd competently to administer my modest mail server which is only used for sites I own, and I'd like to help others as a part of my FOSS pay-forward/pay-back. For now, I'm prepared to spend a lot of time with the docs, reading and writing. At some point I'd like to do more with the package, but while I'm learning I want my newcomer experience to benefit the docs for others.

So for the purposes of this doc site, I'm hoping we can make it a first-class citizen, along with the software, encouraging issue reports, contributions, discussions, etc.

As an example of a simple discussion I'd like to have on this, I'd like to know what you think about moving the Architecture page up in the menu above the Writing Rspamd rules. I'd like confirmation on that before doing it. There are many more such trivial discussions that don't seem to require an Issue/request. Others do. I don't know if I should ask things like this in IRC, a mail list, gitter, Telegram, etc. If questions about docs are asked in the doc repo, we'll have them in one place for easier reference.

Thanks again for your time ... chatty discussions like this will subside as we move forward. :)

[TonyGravagno]

BTW, I have read through Pieter's blog series a few times and agree with his perspective. When I saw that some of his 2018 notes were out of date, and he expressed confusion in his notes, I made changes to the docs. Three years have passed since he wrote that and the time now where such anomalies are being addressed. I hope we can reach out to authors to inform them of updates, and invite them to inform us of their articles so that we can reciprocate with links from the docs. This all happens through Discussions and Issues.

Since we're here, I've written a bunch of JQ queries that parse the output from rspamadm configdump, for example to show all of the enabled symbols for a group, and the groups associated with specific symbols, and symbols related to specified description text. I'll blog this stuff, and I believe the repo should invite material like this to be linked from the docs.

[TonyGravagno]

bump?

[vstakhov]

Yes, sure. There is a section about examples of Rspamd usage where this type of links/material might fit. Or even some entry in the FAQ as a centralised document about various tips for using Rspamd.

[TonyGravagno]

Cool. As I go through docs I'll note where references back here would be of value. Really, I don't think there is much to do on-site - just add a little encouragement for peeps to come here with comment on the docs:

I think it's time for this doc repo to have a ReadMe too. I'll experiment with that to ensure it won't interfere with site publishing.

When convenient, please create a Discussions area here as well. After some of this is setup we can being referring people to the resources in the chat groups, tickets, repo Discussions, etc.

Thanks!