Add x-super-properties header for user accounts

[wavedevgit]

As explained in the first commit description, the x-super-properties header is an important header that if not present may cause discord to flag requests or even worse ban/restrict accounts.
The added code uses this source to add headers that match a normal browser using the discord web client.

possibly closes #1497

[xkti]

Insane this was never implemented until now.

[96-LB]

Have you tested this, and if so does it actually make a difference?

Base-64 decoding the headers you used at https://gist.githubusercontent.com/MinerPL/731977099ca84bef7ad0a66978010045/raw/stable.headers.json (which, by the way, should maybe not be grabbed from an external link), there seem to be tokens unique to specific client instances. Here's a snippet:

"client_launch_id":"970428f4-9ff1-4edf-8265-0b2ce0f5305d","launch_signature":"034d612f-6325-47ca-9f50-f010e8820452"

I worry that using hardcoded values here could either be meaningless or cause more flags.

[dolfies]

You're right that super props are essentially required to send alongside requests, but those values don't seem to be updated.

You guys are free to use my API at POST https://cordapi.dolfi.es/api/v2/properties/web to get automatically updating props :)
User agent and sec headers can be derived from there, and the rest of them can be hardcoded.

[wavedevgit]

You are right, I'll update the code right now to make the values unique.

[wavedevgit]

Btw, I did not test if this code works, as I currently have no way of doing so on Linux. Please verify that it works without any issues.

[dolfies]

This is still problematic, versions are out of date

[wavedevgit]

oh true, I guess i'll use the api you sent, thankss

[Aeplet]

Crazy discord found some shit

[wavedevgit]

I updated it to use dolfie's api

[aamiaa]

FWIW you're making a GET request to his api, while he suggested a POST request, which will give you the xsp data without the need to construct it yourself.

[wavedevgit]

I didn't read that correctly, I'll update the code right now

[wavedevgit]

Done

[AlmightyLks]

I would not see it as wise to rely on a random web api to generate a user agent and x-super-properties, if it can be done by DCE itself easily

[chardidathing]

I would not see it as wise to rely on a random web api to generate a user agent and x-super-properties, if it can be done by DCE itself easily

Generating your own properties seems weird too, I feel like that's very easily detectable :P

[chardidathing]

Re-posting my message from the issue here so you can see my impl, I feel like getting the individual users properties could be a bit more beneficial (Webview2 container or something?) - This would also handle getting the token, which might be a benefit for less technical users.

I've started work on my own, which is contained in a browser extension, it fetches/replays the headers from the current session, so detection should be a lot less possible (fyi @wavedevgit hard-coded headers will most likely get detected relatively quickly)

The current stable version is just scraping the messages directly from the page, with auto scrolling and such.
https://github.com/chardidathing/wumpwump-dl

I have an experimental branch I'm currently working on that interfaces with the API, replaying the headers from the users current session, I haven't tested with higher speeds yet but I can currently do around 100 messages a second (6000 a minute), with not a single rate limit or such.
https://github.com/chardidathing/wumpwump-dl/tree/feature/direct-requests

[Copilot]

Pull request overview

Updates DiscordClient request behavior to more closely mimic a real browser (primarily for user-token requests), likely to reduce Discord anti-bot/anti-scraping detection.

Changes:

• Added GenerateLaunchSignature() and GenerateUuid() helper functions near the top of DiscordClient.cs.
• Added logic in DiscordClient.GetResponseAsync() to fetch browser-like header values from an external API and attach them to outgoing requests for non-bot tokens.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[aamiaa]

I feel like there are a lot of guesses/assumptions being thrown around here

I would not see it as wise to rely on a random web api to generate a user agent and x-super-properties, if it can be done by DCE itself easily

I wouldn't exactly say it can be done "easily". You'd need to get the up-to-date client build number, which requires parsing discord's js files (whose format can change at any time). Of course it can be done, but it might be annoying to maintain.
The api is trustworthy, it's made by a known dataminer (same person who reversed the launch_signature implementation that this pr uses), and is used in projects like dpy-self.

Generating your own properties seems weird too, I feel like that's very easily detectable :P

It's not detectable as long as you do it correctly. It's what Dolfies's api does after all.

I feel like getting the individual users properties could be a bit more beneficial (Webview2 container or something?)

Why would it be more beneficial? As long as you generate valid ones there is virtually no difference. Extracting them from the webview could be problematic too.
Also I assume the user would have to login to their account within the webview, in which case it could appear suspicious to them.

Btw, for your extension, it could be easier to grab and use discord's existing RestAPI class, which includes all necessary headers by default. Intercepting request headers sounds like an overkill.

[chardidathing]

Btw, for your extension, it could be easier to grab and use discord's existing RestAPI class, which includes all necessary headers by default. Intercepting request headers sounds like an overkill.

Yeah I've only just realized this after the fact :P

It's fully working at the moment, I'll clean it up when I have some time over the next few days :>

[tung-lee]

anyone try this?

[wavedevgit]

Someone please try if this works lol

[jamesrfreitas]

Guys, I apologize for asking a beginner’s question.
I’d like to know if the DiscordChatExporter program has been updated to fix the issue that could get your Discord account banned.
Thanks for your patience—I don’t know practically anything about technology.

[wavedevgit]

Guys, I apologize for asking a beginner’s question. I’d like to know if the DiscordChatExporter program has been updated to fix the issue that could get your Discord account banned. Thanks for your patience—I don’t know practically anything about technology.

I can't say for sure, but this pr should fix the issue!

[jamesrfreitas]

Guys, I apologize for asking a beginner’s question. I’d like to know if the DiscordChatExporter program has been updated to fix the issue that could get your Discord account banned. Thanks for your patience—I don’t know practically anything about technology.

I can't say for sure, but this pr should fix the issue!

Thank you for your kind responses. I’ll see what I can do.

[svghubio]

@wavedevgit Your code seems to have been AI generated and doesn't even build:

Restore complete (1.4s)
  DiscordChatExporter.Core net10.0 failed with 1 error(s) (0.6s)
    C:\Users\User\DiscordChatExporter\DiscordChatExporter.Core\Discord\DiscordClient.cs(79,21): error CS0106: The modifier 'static' is not valid for this item

Build failed with 1 error(s) in 2.1s

[wavedevgit]

I fixed this issue (yes the code was made using ai since I dont know csharp but I reviewed it to ensure quality of code)

[svghubio]

@wavedevgit Still doesn't seem to be fixed:

Restore complete (0.8s)
  DiscordChatExporter.Core net10.0 failed with 1 error(s) (2.3s)
    C:\Users\User\DiscordChatExporter\DiscordChatExporter.Core\Discord\DiscordClient.cs(106,40): error CS0103: The name 'cachedHeaders' does not exist in the current context

Build failed with 1 error(s) in 3.2s

EDIT: Building after fixing the variable at line 106.
Might want to add a check for an existing kv.Key though perhaps?

                    foreach (var kv in _cachedBrowserHeaders)
                    {
                        if (!request.Headers.Contains(kv.Key))
                            request.Headers.TryAddWithoutValidation(kv.Key, kv.Value);
                    }

[wavedevgit]

Can you please check now if it works? thank you btw

[svghubio]

@wavedevgit Probably not gonna work, since there is no encoded, browser_version, etc. in the JSON output, which is the response from https://cordapi.dolfi.es/api/v2/properties/web:

{
  "client": {
    "type": "web",
    "build_number": 510733,
    "build_hash": "2fcef2abc770637abb6413c16e81aa7400c70c91",
    "release_channel": "stable",
    "version": null,
    "electron_version": null
  },
  "browser": {
    "type": "Chrome",
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
    "version": "146.0.0.0",
    "os": {
      "type": "Windows",
      "version": "10"
    }
  }
}

Chances are this code won't really work and get my account banned, so I don't want to try any further.

[wavedevgit]

It's a POST request not a GET request
use this in the terminal to test the endpoint:

curl -X POST https://cordapi.dolfi.es/api/v2/properties/web

[AlmightyLks]

@aamiaa

I wouldn't exactly say it can be done "easily". You'd need to get the up-to-date client build number, which requires parsing discord's js files (whose format can change at any time). Of course it can be done, but it might be annoying to maintain.

The up to date client build number can actually be parsed / fetched without searching through linked js files

The api is trustworthy, it's made by a known dataminer (same person who reversed the launch_signature implementation that this pr uses), and is used in projects like dpy-self.

dpy-self is made by the same author as the api host, the trust is there because it is their own dependency. It can just as likely go out of date in terms of parsing behaviour as when DCE would do it on its own. The only benefit is that the parsing mechanism can be changed from afar in case something changes in the future, the downside is that the parsing mechanism can be changed from afar and is dependent on a third party.

Again, I don't see any benefit in using the web api in place for an algorithm / library.
And "might be annoying to maintain" is kind of the entire mantra when working with the discord api ;D DCE and alike are volatile projects, things are bound to break sooner or later and need fixing in some places, but no need to hand out control for that

Edit: I shall also add, that I have nothing against @dolfies , for me personally it's all just about outweighing benefits vs disadvantages, and my bells are ringing when hearing that

[aamiaa]

The up to date client build number can actually be parsed / fetched without searching through linked js files

Yeah I've realized after posting that that GLOBAL_ENV has the build number now, my bad

dpy-self is made by the same author as the api host, the trust is there because it is their own dependency

You missed the point. The library + the api have been trustworthy and up-to-date for a long while as can be seen from the commits history and usage. Would you also distrust it if you were to use it as a regular npm install dependency? I assume not, which is why I'm not sure why is there resistance here.

And "might be annoying to maintain" is kind of the entire mantra when working with the discord api ;D DCE and alike are volatile projects, things are bound to break sooner or later and need fixing in some places, but no need to hand out control for that

That is exactly the issue here though. From what I can tell, DCE's maintainer is not the type of person to watch discord's codebase for changes and apply fixes as soon as something breaks (look at how the tool didn't support forwarded messages until 2 weeks ago), let alone apply antispam preventions proactively. I'm not sure they are familiar with discord's antispam/selfbotting practices either, considering the tool has only sent Authorization and nothing else for the longest time.
This is why "handing out control" for this part would be beneficial imo. Otherwise it's gonna rely on reactive PRs like this one from the handful of people who know what they're doing.

[Tyrrrz]

That is exactly the issue here though. From what I can tell, DCE's maintainer is not the type of person to watch discord's codebase for changes and apply fixes as soon as something breaks.

Trust me, I have a fair share of that over at https://github.com/Tyrrrz/YoutubeExplode ;)

Assuming the changes boil down to updating request headers or their values, I don't see why the community wouldn't be able to handle this.

I'm not sure they are familiar with discord's antispam/selfbotting practices either, considering the tool has only sent Authorization and nothing else for the longest time.

I was familiar to the extent that was necessary for things to work, and, prior to recently, nothing beyond the Authorization header was required for things to work. Peripheral concerns, such as rate limiting, were always handled too:

• DiscordChatExporter/DiscordChatExporter.Core/Discord/DiscordClient.cs

  Lines 53 to 88 in 5cec162

  	// Discord has advisory rate limits (communicated via response headers), but they are typically
  	// way stricter than the actual rate limits enforced by the server.
  	// The user may choose to ignore the advisory rate limits and only retry on hard rate limits,
  	// if they want to prioritize speed over compliance (and safety of their account/bot).
  	// https://github.com/Tyrrrz/DiscordChatExporter/issues/1021
  	if (rateLimitPreference.IsRespectedFor(tokenKind))
  	{
  	var remainingRequestCount = response
  	.Headers.TryGetValue("X-RateLimit-Remaining")
  	?.Pipe(s => int.Parse(s, CultureInfo.InvariantCulture));
  	var resetAfterDelay = response
  	.Headers.TryGetValue("X-RateLimit-Reset-After")
  	?.Pipe(s => double.Parse(s, CultureInfo.InvariantCulture))
  	.Pipe(TimeSpan.FromSeconds);
  	// If this was the last request available before hitting the rate limit,
  	// wait out the reset time so that future requests can succeed.
  	// This may add an unnecessary delay in case the user doesn't intend to
  	// make any more requests, but implementing a smarter solution would
  	// require properly keeping track of Discord's global/per-route/per-resource
  	// rate limits and that's just way too much effort.
  	// https://discord.com/developers/docs/topics/rate-limits
  	if (remainingRequestCount <= 0 && resetAfterDelay is not null)
  	{
  	var delay =
  	// Adding a small buffer to the reset time reduces the chance of getting
  	// rate limited again, because it allows for more requests to be released.
  	(resetAfterDelay.Value + TimeSpan.FromSeconds(1))
  	// Sometimes Discord returns an absurdly high value for the reset time, which
  	// is not actually enforced by the server. So we cap it at a reasonable value.
  	.Clamp(TimeSpan.Zero, TimeSpan.FromSeconds(60));
  	await Task.Delay(delay, innerCancellationToken);
  	}
  	}

• DiscordChatExporter/DiscordChatExporter.Core/Utils/Http.cs

  Lines 58 to 70 in 5cec162

  	// If rate-limited, use retry-after header as the guide.
  	// The response can be null here if an exception was thrown.
  	if (args.Outcome.Result?.Headers.RetryAfter?.Delta is { } retryAfter)
  	{
  	// Add some buffer just in case
  	return ValueTask.FromResult<TimeSpan?>(
  	retryAfter + TimeSpan.FromSeconds(1)
  	);
  	}
  	return ValueTask.FromResult<TimeSpan?>(
  	TimeSpan.FromSeconds(Math.Pow(2, args.AttemptNumber) + 1)
  	);

If you had reasons to believe that was insufficient, prior to the recent policies Discord introduced, then I must've missed it because I didn't see any issues/discussions raised by you.

This is why "handing out control" for this part would be beneficial imo. Otherwise it's gonna rely on reactive PRs like this one from the handful of people who know what they're doing.

Oh, I would love to delegate control over this to somebody else, but unfortunately control doesn't come without responsibility. And it's hard to take responsibility over an API, whose source code I haven't even seen (is there a GitHub repo somewhere?).

It might be that we end up using that API in the end, but right now it's only clear what overhead it brings and not entirely clear how much effort it saves.

[svghubio]

@wavedevgit My apologies, it is indeed a POST request, although user_agent still does not exist; browser_user_agent is used instead. Here is a patch I made on top of your code. I've only tested a modified version of this with the hardcoded parameters I took from my own browser session, though.

0001-Fix-adding-extra-request-headers.patch

Thanks for your contributions.

[wavedevgit]

Thanks @svghubio for the fixed version

[AlmightyLks]

But why 😄

[wavedevgit]

It's just to make sure the headers are added before the request is made (because of the async task)

[AlmightyLks]

Task.Run doesn't help in any way with that. The request goes through only once Http.Client.SendAsync is executed. So the code before was already safe and wasn't at risk of running the request without the headers.
You force a background Task to be created via Task.Run and immediately await it, which effectively is a no-op here, with no other benefit 😄

Edit: Unless I am missing something that you are seeing

[svghubio]

I initially proposed this code. Sorry, I'm not the best at C# 😅
All I know is that with this await Task.Run(async () => it executes per every request just fine for me. Perhaps it could be omitted.

[wavedevgit]

Hm, should I revert that change?

[AlmightyLks]

Yeah

[wavedevgit]

Okay

[unknownsrc]

@aamiaa

I wouldn't exactly say it can be done "easily". You'd need to get the up-to-date client build number, which requires parsing discord's js files (whose format can change at any time). Of course it can be done, but it might be annoying to maintain.

While this is true, is it really a good enough reason to put the working of the whole project behind one single API that might, at some point in the future, without any prior notice, go offline? I say hardcoding a client_build_number, and only using the API to pull the latest one makes the most sense. The rest of the properties can be easily generated.

Also, not sure about the privacy aspect of forcing every single client to send a request to an unknown API that they don't even know they're reaching.

[dolfies]

The point of the API (at least for my use) is to make it easier to maintain props without having to push a release for every new field/minifier change/etc. It can definitely just be done in the project itself and should definitely be done as a fallback either way in case the API goes down for any period of time.

I was just offering it as an option, it's up to the maintainers as to what they actually end up using :p

[tretretrer]

Is there a released compiled version with this fix anywhere ? i see you want us to test it, it might make it easier.

[Tyrrrz]

@tretretrer you can take the dev build produced from this branch: https://github.com/Tyrrrz/DiscordChatExporter/actions/runs/23216779809?pr=1507

[tretretrer]

thanks ill try right now