get_profile status:410 Error

[batuhanmtl]

---

KeyError Traceback (most recent call last)
Cell In[8], line 1
----> 1 api.get_profile(profiles[9]['urn_id'])

File ****site-packages\open_linkedin_api\linkedin.py:747, in Linkedin.get_profile(self, public_id, urn_id)
745 data = res.json()
746 if data and "status" in data and data["status"] != 200:
--> 747 self.logger.info("request failed: {}".format(data["message"]))
748 return {}
750 # massage [profile] data

KeyError: 'message'

[danymajeed]

Hey, did you happen to find a solution for this?

[batuhanmtl]

Hey, did you happen to find a solution for this?

Unfortunately, I couldn't find a solution. The endpoint used by this method must have changed.

[davidbernat]

A brief inspection of the View Source and Network appears to show Voyager shifted to a graphql? implementation.

Users at linkedin-api began having the same problem.
nsandman/linkedin-api#10

If you follow their procedure for finding new API endpoints you see consistently the change in URL form.
https://github.com/nsandman/linkedin-api?tab=readme-ov-file#to-find-endpoints

This is very, very bad for users, and I question their motives at LinkedIn. A thorough search for downloading LinkedIn data repeatedly finds the same three or four useless Reddit.com posts, and not even one browser extension seems to pop up as an alternative for what is obviously a high return use case. The official LinkedIn documents promise "Your LinkedIn data belongs to you, and you can download an archive any time or view the rich media you have uploaded." but on all four occasions that I tried their process over the last two years the results were the same: a few hundred KB of text data which contains none of the images or videos associated with the posts. Their promise to "view the rich media" also leads to a page which contains only the post text with a grey'd out download tab for one image apiece; no organization, no capacity to reconstruct the user posts or timelines. This is very, very bad for users, and mark my words this is the beginning of a new kind of assault of data on the web.

At this point in time zero solutions appear to exist for the most basic cloud-based social network, and I charge intentionally.

I would love to be wrong and hotheaded here but we rowed this boat for two and a half years already. The lists exist. Trust me.

[andriyshevchenko]

@davidbernat @batuhanmtl @danymajeed @EseToni Do you guys think this approach will work?

1. Open Chrome DevTools.
2. Set 'voyager' filter in the Network tab.
3. Capture the HTTP requests via Dev Tools and export as HAR.
4. Feed the HAR archive (which is basically a text file) to some AI (Claude, Cursor)
5. Ask to properly update endpoints / figure out the new schema.

Or it's just a waste of time given that LinkedIn changes its endpoints very frequently?

UPD: I've had a plenty of success with it. 410 errors gone.

[batuhanmtl]

@andriyshevchenko Thanks for sharing this approach 🙌

I followed a very similar workflow: capturing Voyager requests and then inspecting/updating the endpoints based on the new schema. After updating the calls accordingly, the 410 errors disappeared and get_profile started working again on my side.

Additionally, get_profile_skills was previously returning an empty list; applying the same approach to that method also resolved the issue.

So at least for now, this approach definitely works. Thanks for sharing 👍

[gtkacz]

@andriyshevchenko @batuhanmtl how so? All I see are GraphQL requests

[batuhanmtl]

@andriyshevchenko @batuhanmtl how so? All I see are GraphQL requests

Analyzing the HAR file is very helpful for understanding LinkedIn's new API structure. I especially recommend focusing on these details:

Endpoint Changes:
Examine the URLs of the requests made in the HAR file. Try to find the differences between the old and new endpoints (e.g., Dash-based endpoints instead of /profileView).

Query Parameters:
Observe how parameters like decorationId, q, and memberIdentity are used and what their values ​​are.

Decorator IDs:
Note the decorator IDs used in successful (200 OK) requests. These define LinkedIn's data schema and the fields returned.

Request Headers:
Check the custom headers used in requests (e.g., CSRF tokens, API versions, cookies). LinkedIn sometimes rejects requests without these headers.

Response Format:
Look at the structure of the returned data. Are there new fields like elements and included? Is the data flatter? How is relational data organized?

Version Numbers:
The numbers at the end of Decorator IDs (e.g., -185) indicate LinkedIn's API version. Compare whether different numbers were used in different requests.

[gtkacz]

@batuhanmtl can we please not post LLM generated comments here? Could you post your code changes or even create a PR? I still only see GraphQL calls

[andriyshevchenko]

@gtkacz @davidbernat @danymajeed @batuhanmtl I was able to implement a plenty of functionality using this approach like: scheduling/unscheduling posts, sending/withdrawal of connection requests. You can take a more deep dive here https://github.com/andriyshevchenko/linkedin-voyager-mcp

UPD: it seems to me LinkedIn changed something about their GraphQL API.

get_profile broke again after I fixed it 🥲

[davidbernat]

@andriyshevchenko noted, but this is a $35 billion company with twenty years of history and user data. I want a button that says "Here is item of content you ever uploaded, in an HTML and zip to view at your leisure, and every telemetry item we collected on what you interacted with. Thank you for being a valued asset to our growing enterprise. There is no fine print."

The Voyager API is an ad-hoc unofficial substitute, which broke, and now we are discussing one-off parts that kind of work?
I respect the hustle. How close to nominal are we?

[andriyshevchenko]

@davidbernat depends on your goal, if your goal is to download all your posts, I doubt it's possible. They are replacing GET GraphQL requests with SDUI - server driven UI, which is basically a streamed HTML components, server-compiled. Impossible to parse or scrap, because everything is obfuscated. GraphQL is still "kind of works" when it comes to POST requests: add connection, send message etc.

[arunpt]

@andriyshevchenko Any idea on how do we scrape if they moved completely to SDUI?