State of the REST API for upload

[JbIPS]

Question

Which REST API should I use for book data upload?

Additional context

The wiki points to

POST https://openlibrary.org/api/new.json

but the OpenLibrary python client seems to use

url = self.base_url + '/books/add'

and the wiki is...well not very insightful:

How to upload

• Create an archive.org account: https://archive.org/account/signup
• FTP the files to catalog-upload.archive.org using the username (most likely your email address) and password you just created. This doesn't seem to be working right now.
• Contact us and let us know if you have uploaded something.

For context, I'm looking to help a FOSS app OpenReads to upload user-written description when the book doesn't exist in OL (see mateusz-bak/openreads#85)

Issue resolution criteria

A clear wiki entry/webpage/REST API schema for data upload

[scottbarnes]

@JbIPS, sorry for the horrifically late reply, and I'm sorry for the poor documentation around the APIs.

To some extent, it depends a bit on what you want to do. If you'd like control over the specific fields, the API located at /api/import may be the best. That's documented (twice apparently...) here and here, with the latter being perhaps a bit more complete. I'll look at trying to consolidate these pages, as I didn't know that endpoint page existed.

If you're using Python, the openlibrary-client directions may also be useful.

/api/import will require a bot account, though that's not an especially hard hurdle to overcome. If you're still interested in this, do you have any ideas how authentication might be handled? Are you thinking of having one account that would simply update descriptions on behalf (whatever that means) of OpenReads patrons?

Are you the Internet Archive Slack?

[JbIPS]

Hello @scottbarnes

Don't worry for the delay, I know that kind of project (and probably others) can be very time consuming.

I think I'll start by tinkering with /api/import/ with a bot account and I supposed that bot could be transfered to OpenReads so patrons won't have to create an individual account to contribute. If I find a clear enough walkthrough, I'll come back to you for the documentation, it can be a good start 😉

And thank you for the link to the slack, I'll go check that!

[scottbarnes]

@JbIPS , I realize I should have mentioned if you start experimenting in the local development environment, the default openlibrary@example.com account will work. That may be the best way to start, and will make it easier to get the bot account approved because you'll be able to determine exactly what you want to do, and you'll be able to experiment against the local development environment rather than the production data.

Also, if you know which Edition or Work you want to modify, the openlibrary-client (this link is different from the previous openlibrary-client link, and is about using it for more than just imports) is probably the way to go, though that assumes you're using Python.

There's also this client in C#. Between that and the Open Library client, hopefully you get a good sense of what to do if you're using another language.

Both of these are mainly working with the the JSON endpoints, and you can see how saving is done here.

/api/import is meant for things that may not be in Open Library, and not all fields will be updated via that API, so depending on how you plan or want to do things, I realized that API not be the great. But if you're working in Python and know the Edition or Work you want to modify, openlibrary-client makes it really easy and there are examples on in the openlibrary-client repository.

[JbIPS]

Thank you for the advise. I tried to play with the development docker earlier but got 401 on /api/import/ with the default account (I think it's openlibrary@example.com). Maybe I missed something in the cookie...

My use case, at first, is to create a book entity if it does not already exist in OL, so Edition/Work modification won't be enough. The android app is in Flutter, so it will be a JS implementation, maybe it will be an opportunity to do a JS client 😊.

I'll take a look at the bot repository, some seems to be a good start for my usecase.