Replies: 3 comments 5 replies
-
|
Here's hoping HMRC figure something out. It looks like HMRC only took closed source software into consideration when designing their system/API. It's a shame. Being closed source wouldn't stop someone determined and skilled enough from extracting those keys and misusing them either.. Perhaps the solution would be for HMRC to simply set the system so that people who are submitting tax always get their own set of API keys, associated with their "Tax account". Thus, it becomes the users responsibility to register and then add those keys to the application (much like the sandbox keys, except being associated with the tax account itself instead.) The way they are doing it now just feels like their intent was to get commercial companies to have an advantage to open source software (thus getting their backhander, and helping the companies earn more) C'mon HMRC, sort it out. |
Beta Was this translation helpful? Give feedback.
-
Hmm, yeah and if when it comes time to get production credentials, if you had the ability to rather then select one of your sandbox applications but rather select an application that has already gone through the process (perhaps that would be something the application author would enable allowing), basically linking it your account. Hopefully that would also alleviate the need to go through the whole 'production credentials' process. |
Beta Was this translation helpful? Give feedback.
-
|
This thread has confirmed my suspicions about the process being a little complicated if you're open source. It's a little surprising only because they make nearly all their own stuff open source now. I've found their documentation to be easy to use, and I've been happy with the test reponses so far. (The "stateless" aspect of the SA testing environment is obviously going to be unhelpful, and I'm just getting to that.) Obviously I'm doing my own, pulling stuff from Google Sheets in my case. It's private on github at the moment, just to avoid visible churn until it's more useful. |
Beta Was this translation helpful? Give feedback.
-
|
I think it's not because of wanting to help companies earn more, but rather because no-one seems to have realised that if you do an effective API, programmers who are so inclined and capable will write their own in some cases, rather than say subscribe to Xero. |
Beta Was this translation helpful? Give feedback.
-
|
So what is your current opinion on the feasibility of doing MTD using open-source? Clearly going through a 1.5 year process to get production credentials is not practical. Which essentially leaves people who want to use open-source accounting software no other option than to write out the data into an excel sheet and feed it through closed-source bridging software? |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Certainly I got the feeling that they had simply not considered anyone doing an Open Source MTD application! There is one other open source MTD project I'm aware of who also has production credentials. gnucash-uk-vat. As he mentions, if you were to use either of our applications, hopefully the fact they have gone through this process will make it easier to get credentials to use with them. For a new Open Souirce project, I have no idea if the process will be any better now... IIRC the main issue is with the credentials. For a proprietary application the credentials would be embedded within (in some way that makes it hard to simply extract them). For an open source project this isn't really possible, hence why they'd want any users of such programs to apply for their own credentials. |
Beta Was this translation helpful? Give feedback.
-
|
gnucash-uk-vat maintainer here. I previously had more info on how to sign up using gnucash-uk-vat on the Github repo but HMRC dev support team asked me to take it down. We're definitely exercising a path they hadn't planned for. I'm not familiar with ITSA, I guess it's the same auth mechanism. I have a secret production key, but I can't share it, so it means anyone else wanting to use the software has to go through the same process I went through to get to 'production'. Which doesn't 100% make sense to me if people want a standard 'user' signup path. I wonder if there's another way... if I hosted a site where could people could authenticate through to HRMC and get an auth key back without knowing my secret key. It would only need to deal with the auth part, and could be used with the open source code. |
Beta Was this translation helpful? Give feedback.
-
Exactly! Our experience shows that open source software simply didn't enter their conciousness when dreaming this whole thing up.
You know, that's an interesting idea. In fact HMRC even seem to have a connection method that covers this scenario. Although this would perhaps be a more hybrid DESKTOP_APP_VIA_SERVER/DESKTOP_APP_DIRECT thing, the Fraud Prevention Headers being an added complication... but it could maybe work... |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
tl;dr Production credentials for the Income TAX Self-Assessment API have been granted to itsa & mtd-cli (2021-11-19)
So this is about trying to put an open source application (well two actually) through the HMRC Making Tax Digital procedure to get production credentials.
This relates to mtd-cli & itsa but I've put it here as it's the underlying library.
This process is taking way longer than expected, the main issue seems to be HMRC don't know what to do about the client/secret tokens and open source projects as these tokens are meant to be kept secret.
The following is a time line of events
2020-05-29 (May 29th 2020)
I've already thought of this problem myself and posed the question to HMRC about how to handle the tokens.
Reply:
OK fair enough...
2021-03-10 (March 10th 2021)
So it's sometime and a fair bit of hacking later, I'm starting the procedure for acquiring production credentials. Between March 10th and March 16th there is some back and forth over email as they have some questions and where I explain in more detail about what I've done. On the 16th I did mention explicitly that this is all open source
The licenses are also mentioned on the applications details pages in the HMRC developer hub, but perhaps I should have been more loud about this.
2021-05-06 (May 6th 2021)
This is the day of a live demo with HMRC. Unfortunately things went downhill pretty quickly.
I wasn't long into the demo when they picked up on a problem with the tokens. I was doing an
on route to creating an employment for the demo (showing them the whole thing from start to finish). Of course as part of that is telling the application what the client_id & client_secret is.
That immediately triggered concern from them that these tokens where somehow being made public. I explained that is not the case, they are only stored locally etc.
They then queried about how this can work, every time someone wants to use these applications they would need to go through this whole process and said that was not feasible/scalable.
At that point I informed them that is essentially how I had been previously told it would work.
At this point the demo was terminated and they wanted me to forward a copy of the email from HMRC/them that said that.
Email forwarded.
2021-05-10 (May 10th 2021)
Got a reply back
2021-05-24 (May 24th 2021)
A couple of weeks go by and no response.
I ping them...
2021-06-03 (June 3rd 2021)
I ping them again.
2021-06-11 (June 11th 2021)
Finally get a reply back
Hmm, 'in house product', well yes, but it's more the fact it's open source isn't it?
At this point I also point out that in fact, another open source project has managed to acquire production credentials.
Which I put to them but no direct response.
I'll also note here that this project seems to have been told basically the same thing I was
2021-06-24 (June 24th 2021)
I ping them again...
2021-07-02 (July 2nd 2021)
Still working on it...
2021-07-27 (July 27th 2021)
Ping again...
2021-08-10 (August 10th 2021)
Pinged again...
2021-09-09 (September 9th 2021)
New email sent trying to kickstart things again...
2021-09-17 (September 17th 2021)
So that seemed to do the trick. Had a short but somewhat productive chat with HMRC today.
After verifying a few details, I just need to run itsa to generate some endpoint logs for them to look at (again) and then it sounds like I should be able to get production credentials, for itsa at least.
This is not helped by the fact that the sandbox environment is a bit messed up currently
Inconsistant tax year used in canned responsesInconsistent businessId used in various sandbox endpointsmtd-cli on the other hand may require some more faffing about however, due to also supporting VAT... just like a Star Wars movie, the saga continues...
2021-09-19 (September 19th 2021)
Email sent notifying them of an itsa run for them to check.
2021-10-07 (October 7th 2021)
Between fixes (see above closed issues) at their end and some kludges at mine (businessId fixup for the Self-Assessment endpoints and hardcoding the date period for the "create period" endpoint, although you can now specify that on the command line) I can now do a test run without errors.
Another email sent off to HMRC saying as much with details of test run.
2021-10-08 (October 8th 2021)
Email back from HMRC
Heh, decision had better be, here's your production credentials!
2021-10-13 (October 13th 2021)
Had a (what I feel was a somewhat condescending) email from them, trimmed and some comments below
They seem to be quite hung up on the fact that this is just a personal project.
Woah, OK!
Yes itsa is just a commend line program. though I have tried to make it look nice. But they never even saw any of that. The only thing they say was the initial setup, being done from mtd-cli to create a business for the demo, and what they saw was just coming out of libmtdac, stuff you'd see once (or every 19 months when the OAuth code expires). So they never actually saw what itsa looks like.
Yes, I get the message...
Anyway, I replied and even included a screenshot (shown below) of itsa doing crystallisation, so they could see kind of what it looks like.
2021-11-03 (November 3rd 2021)
So after another three weeks have passed with no sign of anything happening, a rather curt email has been sent asking what's happening...
2021-11-03 (November 3rd 2021)
Reply
Yes, because what I'm doing is of course totally nuts! (sarcasm b.t.w).
It's now Friday night on the 12th and surprise surprise, heard nothing. Guess another email will be getting sent off on Monday, though the only thing I expect from them these days is the run around. Maybe they're hoping I'll just go away!?
2021-11-15 (November 15th 2021)
Pinged them...
2021-11-15 (November 15th 2021)
Always seems to be one more question... Anyway told them for the umpteenth time, only I know my client credentials and if anyone wishes to use this they would need to get their own...
2021-11-19 (November 19th 2021)
So I finally have production credentials for itsa and mtd-cli
mtd-cli only has them for the ITSA endpoints. I need to apply for the VAT stuff separately, lets see how that goes...
Beta Was this translation helpful? Give feedback.
All reactions