RESTfull: patch, delete, soft delete
Working on knowledge base project.
I need to implement delete/undelete article action functionality for APIv2.
Unfortunately there is no strict technical requirements written (or I've missed them somehow :-( )
@Todd had shared this article previously
https://philsturgeon.uk/rest/2014/05/25/restful-deletions-restorations-and-revisions/
And I have this another link from @Ryan
https://www.pandastrike.com/posts/20161004-soft-deletes-http-api/
That is not specs, but food for thoughts.
Main points I get from these article together with my opinion and my experience are :
- We should NOT use DELETE request when we do "soft delete" and switching status of post or article or whatever endpoint model is
- We should use PATCH request when we do change some flag field (like "status" in our case)
I've implemented 2 PATCH endpoints for delete/undelete actions:
In this case I've felt free to accept no extra incoming arguments to come to my endpoints since I have and ID provided by url request itself.
I also don't see any reason to return any content. My expectation is that HTTP STATUS 2XX, in this case 204 should be enough. And some 4XX or 5XX could appear if something went wrong.
But I've got feedback on my PR:
- Every patch in Vanilla takes some parameters, even if it's the one value you're changing.
- Vanilla's APIs output the result of patch actions (e.g. the modified record).
- This seems just about identical to the other method, with the minor change in the value we're setting. Could they be combined and utilize input from the request to determine if it was a delete or restore?
- In a RESTful API, you generally want to avoid using HTTP verbs ( e.g. delete) in your URLs for the sake of avoiding confusion.
#Note: Easy solution to fit to all this "requirements" is to not implement these 2 new endpoints and just reuse existing universal PATCH endpoint to set 'status' field to whatever we need. That would allow to pass code review. So lets assume that we need those new extra PATCH endpoint to exist.
I want to discuss those questions itself with some my personal point of view on them. And I want to get everybody's point of view please for every sub-topic.
Every patch in Vanilla takes some parameters, even if it's the one value you're changing.
Why should any API endpoint "take some parameters" even if it does not need one? To me it sounds weird and almost wrong. In this case we already have an ID provided from the url and there is no need for any extra parameter to be present.
Vanilla's APIs output the result of patch actions (e.g. the modified record).
Same thing here: why should we output an empty array or any other response if status is enough? IMHO that is one of the general ideas of RESTful API - we have various request types and we have various responses when needed. Why should I return modified record if client requested "delete" an item. I feel we only need to return SUCCESS or FAIL response and we can use http status for this purpose.
"Could they be combined and utilize input from the request to determine if it was a delete or restore?"
Short and obvious answer is "yes" but if I implemented that I would go with #Note
But I want to answer by asking another question: is it the best approach?
Honestly I don't have a solid answer or strong opinion on that.
But I have my doubts based on OOP practices, especially the one called "Single responsibility principle" . I see that principle broken all around old Vanilla code.
And my idea is if we have 2 different well defined actions (from UX/client perspective) which have different meaning for the product and for the user, and we are about to use them independently to each other.
- For example we can expect user to have permission to execute one action but not another.
- Or one action will become much heavier than another.
Even if at the starting point they look similar I prefer to have 2 dedicated methods/endpoints and not to mix/aggregate them. And that is why I doubt if #Note is the best approach. Please keep in mind that we don't discuss the DB model adapter when we expect a method/endpoint just to do a small little very specific technical thing like update a column on the DB table. Logic of any endpoint can be pretty complex.
If we go #Note way, then we will get our PATCH method one day to look like:
if ($status === ArticleModel::STATUS_DELETED) {
...
// or we can call another dedicated method instead of do something here
} elseif ($status === ArticleModel::STATUS_PUBLISHED) {
... // some action dependent permission checks and calculations
// or if we call some another dedicated method - what was the reason to aggregate them into common PATCH method?
} else ...
// it can be switch...case instead of if,
// but my opinion it is just something that should not be here
Another argument to keep endpoints independent and not extend an existing one is the simplicity of keeping documentation up to date and keeping RESTful url calls explicit. But not to add more and more arguments/parameters to the same method/endpoint which makes it much more difficult for future developers to get the API functionality. Even now I am a bit confused about the PATCH endpoint we have:
Doc says to me as a developer that this endpoint receives (up to) 2 incoming arguments: knowledgsCategoryID and sort . Reading this I could probably assume: "Ok. I can change article category by using this endpoint". What response I would expect when I want to change the category of article? I would expect something pretty simple: success or failure status. But lets check what I will get in reality according to api doc:
Hmm... To me there is a "bit" more than I would expect from the method with 1 or 2 arguments to patch.
Another problem with "universal" patch is what I’ll call a "semantic" issue. If we accept 2 incoming arguments: knowledgsCategoryID and sort could I send them both? If the request will fail - is it failing on both?
If I add status to that incoming parameters list that will not be equivalent of "delete" action anymore, because it can accept "knowledgeCategoryID" - which means "change category" action from UX perspective. Should client be able to change category when "delete" an article? Where and when application will check and resolve such conflicts?
With the time this universal PATCH endpoint will transform to some monster kind function.
That is why I would prefer to have dedicated endpoints for actions which have UX sense and I prefer to have many small simple methods even if they seem like "brothers" or "twins". It comes from OPP practice with getters and setters when even for small things we do create dedicated methods even when they do almost nothing.
In a RESTful API, you generally want to avoid using HTTP verbs ( e.g. delete) in your URLs for the sake of avoiding confusion.
Generally I am completely agree with this statement or I would call it pattern. Here are some thoughts about that topic, from the same author of the article Todd shared:
https://philsturgeon.uk/rest/2014/05/11/restful-urls-actions-need-not-apply/
Where author compare and guide us through transformation of a SOAP request:
POST /SendUserMessage HTTP/1.1 Host: example.com Content-Type: application/x-www-form-urlencoded id=5&message=Hello!
to RESTfull:
POST /users/5/messages HTTP/1.1
Host: example.com
Content-Type: application/json
{ "message" : "Hello!" }
The key change related to this topic is that the verb Send is gone and id has transformed from the argument passed as a POST parameter to an argument we can get from url. The second transformation IMHO is very important RESTful semantic difference
In my opinion RESTfull urls should be as explicit as possible. The author mentioned for example:
GET /users/philsturgeon/messages PATCH /users/philsturgeon/messages/xdWRwerG DELETE /users/philsturgeon/messages/xdWRwerG
In our case as a developer I would prefer to have some number of explicit endpoints with no extra arguments need to pass in the body:
PATCH /articles/{id}/delete
PATCH /articles/{id}/undelete
PATCH /articles/{id}/publish
Than to have one big endpoint with various combinations of arguments. Where arguments can change the sense of action client (me as developer) want to achieve. A little confusion about delete verb seem acceptable.
Sorry I could not find a laconic way to express myself. I wanted to give some background and share the full picture of why I have all these question.
And the very last topic/question I have related to this discussion: should such taste /style preferences be a blocking thing for PR? should we implement some standards for this? Or can we keep it open and give every developer some freedom about the way he is comfortable implementing some features?
Comments
-
I generally agree with the two pieces of feedback you got, but I also agree with some of the points you make. I don't think they are mutually exclusive.
I would say all of this is true:
- A
DELETErequest would be inappropriate here. - Using
/deleteas an endpoint is confusing. - A "universal"
PATCHendpoint is bad, for the reasons you note. - Sending redundant information is bad, again as you note.
- A consensus on implicit rules in how our API is constructed is important so it doesn't feel like a Frankenstein's monster when folks go to use it.
It seems to me the way to satisfy all of the above would be something like:
PATCH /articles/5/status HTTP/1.1 Host: example.com Content-Type: application/json { "visible" : "0" }The point isn't in the particular word choices I've made there, but rather that we're using one endpoint for both toggle directions, giving us a logical payload choice (visible=0 or visible=1), and making a full response similarly logical to confirm it's now the intended value. It's not a "universal" patch, but nor is it entirely bespoke - it's possible we could add other states in the future or make a more complex status calculation.
Coming to a consensus on "rules" like "Every PATCH request should take input and give a response" is important precisely because limitations like that constrain our design enough to create a coherent experience across our project. In this case, the rule manifests by dictating that a toggle use one endpoint rather than two, since two would implicitly violate the rule (by making further parameters redundant). That kind of emergent design is what holds complex systems together as they grow.
1 - A
-
As an aside, I want to reiterate that the verbiage used in our UI doesn't need to be reflected in our API. Calling it "Delete" in the UI is a design/product decision. What to call it in the API is another matter entirely, and the conventions of a REST API lead us to the conclusion that we probably shouldn't use the word 'delete' anywhere at all, in this case.
0 -
Why should any API endpoint "take some parameters" even if it does not need one?
It's the convention. The API is an interface that is exposed to users. It's important that it be a consistent experience. That means not only consistent with Vanilla's APIs, but APIs, in general. Actions working differently between endpoints is not consistent. The general practice is you send a patch request to an endpoint and the body of the request contains the changes you want to make. Forgoing the body in a patch request is probably going to be counterintuitive for a lot of people and will likely feel wrong.
why should we output an empty array or any other response if status is enough?
When you patch a row, you typically return the altered value. I can send a request, get a 200 and assume it's okay, or I can get the altered row/field in the response and be sure it's been updated. The idea that we'd throw an error if something went wrong is valid in theory, but the confirmation of receiving the altered row makes me more confident the action successfully completed. Again, this is commonplace in existing APIs in and out of Vanilla. I don't think it's a good idea to change the way things work in some areas, just because part of the development team feels differently today than it did six months ago. If there's a valid reason not to do it, then it should be redone everywhere.
It's also worth noting that we use this design in Vanilla for things like discussion updates. For example, you can send a patch request on a discussion record and get the updated discussion in the result. This simplifies the overall number of API requests required and can be utilized effectively by UIs to avoid a subsequent API calls (i.e. a patch request to update the row, a followup get request to grab the updated resource).
But I want to answer by asking another question: is it the best approach?
I don't think the single responsibility principle is broken by managing the deleted status of an article in a single API endpoint ( also: API interfaces are not OOP). I'm certainly not dictating that's what we should do , hence the original question in the PR. But to me, it seems simpler for the client to manage the deleted status in one area and pass a value to toggle the flag. There are fewer items to document, which makes it easier on the user, and fewer items to maintain, which makes it easier on us. Having separate methods on the API controller just seems unnecessarily complicated to me. I'm not saying everything has to go under the umbrella patch operation, but these two extremely similar actions (delete/undelete) could share a lot of code.
Another problem with "universal" patch is what I’ll call a "semantic" issue. If we accept 2 incoming arguments:
knowledgsCategoryIDandsortcould I send them both? If the request will fail - is it failing on both?All of our patch requests are "sparse" updates. You don't need to specify every field that is accepted.
should such taste /style preferences be a blocking thing for PR? should we implement some standards for this? Or can we keep it open and give every developer some freedom about the way he is comfortable implementing some features?
Personal preferences shouldn't be PR blockers, no. But when it comes to the API, which is our product's public interface, it's extremely important to have a consistent experience. That means developers shouldn't implement an interface that functions differently from everything else, just because they think it should be different. Me saying your patch should have a request body, as well as including a relevant body in the response, isn't because I, personally, think it should. It's because it's the way anyone using our API likely expects it to, based on how everything else works.
It's also important to keep in mind that not every comment on a PR is a "blocker". Sometimes they're just questions. Sometimes they're suggestions. Sometimes they're just idle observations.
0 -
In our case as a developer I would prefer to have some number of explicit endpoints with no extra arguments need to pass in the body:
PATCH /articles/{id}/delete PATCH /articles/{id}/undelete PATCH /articles/{id}/publishBut that isn't restful. A restful API doesn't include verbs in the URL, because it uses the relevant HTTP verbs. That is why the example you cited dropped "SendUserMessage" in the SOAP version to just using a post verb in the restful version. You use relevant HTTP verbs against specific resources. "Delete", "undelete" and "publish" aren't resources.
0 -
A restful API doesn't include verbs in the URL
Can I get a link to read about this restriction please?
and what about
GET /articles/{id}/editisn't
edita verb?Me saying your patch should have a request body, as well as including a relevant body in the response, isn't because I, personally, think it should. It's because it's the way anyone using our API likely expects it to, based on how everything else works.
That is the whole my point is. There is your personal opinion and there are opinions of others. For example there is mine. "anyone using our API" - that is me, so to me our swagger documentation is not very clear. That is good to have one than to not to have any, here (I hope) we are on the same page. But my expectations are different from what you have in mind about "anyone". And that is why I completely understand that many of my assumptions could be wrong as well.
That is why I've asked if we should implement some documented standards. Having written standards I or any other developer should not spend time to do research and to make any assumption but just to follow standards.
I have same point of view for "missing" technical requirements. It was not set up if I (or any developer) should use DELETE, PUT, PATCH, POST. I've interpreted that to: developer can make his own decision.
I think we should either have pretty well defined technical requirements for each task or allow developer to apply whatever he feels appropriate in particular situation with respect of team/product/framework standards.
All of our patch requests are "sparse" updates. You don't need to specify every field that is accepted.
Sorry, can't get word "sparse".
If I don't need to specify every field that is accepted. What is the reason for in-schema to exist? And what is the reason of this swagger documentation? Do we have it just for fun?
0 -
It's also worth noting that we use this design in Vanilla for things like discussion updates. For example, you can send a patch request on a discussion record and get the updated discussion in the result. This simplifies the overall number of API requests required and can be utilized effectively by UIs to avoid a subsequent API calls (i.e. a patch request to update the row, a followup get request to grab the updated resource).
I'd like to point out that in the the one UI case we have UI for with an update type action currently (submitting a new revision, modifies the article) we don't actually return back the canonical URL for the result of the POST so the result just gets thrown in the garbage and a separate request is made anyways.
Even if the response had that URL it is insufficient for the next action (viewing the article) because it does not contain expanded field data.
GET /articles/{id}/editisn't
edita verb?This endpoint also currently has no use in our UI. As I understand it does not return the edit body of the current revision. Aside from that there are no edit specific properties, so the endpoint has no use.
0 -
isn't
edita verb?Not in this case, no. An "edit" isn't being performed by that get request. It's retrieving the row with specific values to aid in a patch request (e.g. returning the raw body content).
Can I get a link to read about this restriction please?
The post you originally linked to with the SOAP example (RESTful URLs: Actions Need Not Apply) discusses it.
A verb is an action - a doing term, and our API only needs one verb - the HTTP Method. All other verbs need to stay out of the URL.
0 -
@Ryan links about not using verb is not standards but just opinions with wide explanation why we should follow them. And in general I agree with reasons provided in that article. But in reality that is up to the team to decide and/or to set product/team standards.
Not in this case, no.
Sorry, but you are wrong.
Editis still the verb. You just have your explanation/excuse why we should be OK to use this verb in this particular case.An "edit" isn't being performed
Should I say: " An delete isn't being performed" for my case.?
0 -
I think we should either have pretty well defined technical requirements for each task or allow developer to apply whatever he feels appropriate in particular situation with respect of team/product/framework standards.
I've interpreted that to: developer can make his own decision.
That's where architecting a solution comes in. We had some recommended reading for the "soft delete" action, which illustrated some common approaches. We know the basic acceptance criteria for the functionality. It's more on us to define the technical criteria, but we aren't developing an purely internal component. This API will very likely be used by customers. It is crucial we consider common API practices, both in Vanilla's existing endpoints and in contemporary API design. It has to feel like using the rest of the API v2, just as much as it has to feel like a restful API (versus a SOAP API). Consistently following restful API best practices makes the API feel more intuitive. These endpoints cannot be designed according to however each individual developer feels about implementing them.
Sorry, can't get word "sparse".
If I don't need to specify every field that is accepted. What is the reason for in-schema to exist?
By "sparse" I mean: you can send one, all or only a few of the fields in most of our patch requests.
From a documentation point of view, the "in" schema exists to let you know what the endpoint accepts. It is also responsible for validating the input request.
And what is the reason of this swagger documentation? Do we have it just for fun?
The Swagger documentation is useful for anyone who wants to see what capabilities are available to them via the API. It automatic generation certainly has its bugs, but we are working to improve it. If you find anything specifically wrong, I highly recommend filing an issue for it (if one doesn't already exist).
0 -
Should I say: " An delete isn't being performed" for my case.
I think you're trying to twist this into semantics, which is counter-productive. We want to avoid verbs in our API: "doing words". This is per restful API best practices. Again, we want our API to feel like a restful API by adhering to these. You are advocating merging actions into URLs, which makes this more of a SOAP-style API.
A get request URL containing the word "edit", that isn't "doing" anything beyond a normal get request is not a "doing word". A patch request containing the word "delete" in its URL and is performing a soft delete is certainly a "doing word".
0 -
My 2 cents:
PATCH /articles/{id}/softDeletedsoftDeleted:bAddresses:
- The verb problem
- The clarity with DELETE
- If you want to "restore" just put softDeleted = 0
- Anyone reading the following will understand
if ($record['softDeleted']) {continue;}(I can't codeblock in a list!)
0 -
We want to avoid verbs in our API
We want != We must
Half of this post questions is about to get all your/our wants written either in standards or requirements. You are ignoring this part of my question completely. I don't know why.
It has to feel like using the rest of the API v2, just as much as it has to feel like a restful API (versus a SOAP API)
Unfortunately or fortunately I have different feel. Which means some other developers could feel different too. That is why we should have standards and/or requirements written.
From a documentation point of view, the "in" schema exists to let you know what the endpoint accepts. It is also responsible for validating the input request.
lets take a look again:
Another problem with "universal" patch is what I’ll call a "semantic" issue. If we accept 2 incoming arguments:
knowledgsCategoryIDandsortcould I send them both? If the request will fail - is it failing on both?How many incoming arguments this endpoint accepts up to 2 or up to unknown all fields on database table?
0 -
We want != We must
That is true. We don't have to make a restful API. We want to. We don't have to make it follow best practices. We want to. We want to do this, because other professional APIs follow the same patterns and practices. They are patterns and practices that have been tried and tested for years, by companies much larger than our own. They are existing, valid solutions to common problems. If we say we have a restful API, there are expectations that it functions in a certain way. We don't have to meet those expectations. We want to.
There isn't a stone tablet of "Best Practices for RESTful APIs". If you want to read an RFC, you can probably find a list of "musts" and "must nots", but beyond that, you'll find the recommendations are largely a collaborative effort by the global developer community. We rely on the insight and experiences of others, so we don't have to make the same mistakes.
One of the biggest players in the knowledge base space is Zendesk. Their restful API docs are public and available for review. You can checkout their Articles endpoint page for an example. You'll notice none of their URLs include "doing words". This is the same across all their endpoints, near as I can tell. They rely on HTTP verbs, not "actions" in the URL. This is what we'll be competing against and this is the type of API interface professional developers (especially from companies we poach) will expect. Like Zendesk, we can follow an established, widely-accepted pattern for designing APIs, or we can try to make things up as we go. I do not think we want to go with the latter.
How many incoming arguments this endpoint accepts up to 2 or up to unknown all fields on database?
It accepts two. You can give it one, if you want. Either one. The endpoint doesn't care how many fields you send it. It'll update the target record with the values you sent. If you send it fields that it doesn't accept (i.e. anything other than
knowledgeCategoryIDandsort), the input schema will remove them before the save operation is performed.0 -
I feel
statusesis probably the better word here, because we are likely going to have more statuses such aspublished.0 -
How about "remove" and "restore"? or is that stepping on other endpoints?
0 -
0
-
Pretty much all of the get verbs are specific to getting data to perform an action. I said this before, but they are not "doing words". I'm not creating a quote when I access /discussion/{id}/quote, I'm getting a discussion row in a format that's more compatible for quoting. I'm not editing a comment when I access "/comments/{id}/edit", I'm getting the fields necessary for an edit (e.g. the raw body contents, instead of the rendered body contents). The act of creating a discussion with a quote is still a post to "/discussions". The act of editing a comment is still a patch to "/comments/{id}".
The put request to "/users/{id}/ban" is writing to a specific field. The user's row has a "ban" flag. It's part of the resource. Similarly, "/categories/{id}/follow" is updating the user's "follow" status of the target category. It's part of the user's category data. You can unban a user or unfollow a category with these same endpoints by changing the request body. You're directly setting fields on the resources. You aren't banning a user by requesting "/users/{id}/ban". You're updating the user's ban flag. You aren't following a category by requesting "/categories/{id}/follow" . You are updating the "follow" flag in your personal category data.
Posting to "/users/{id}/confirm-email" is not ideal. Given the chance to do this over again, I would maybe rename it to "/users/{id}/email-confirmed". The actual field is "Confirmed" in the database, but that name is probably not intuitive to most users.
Posting to "/media/scrape" is not a typical scenario. In this case, we're essentially managing a virtual resource. It scrapes external page data. We had a lot of internal discussion of how to go about creating the interface for this operation and this is ultimately what we decided on. This is definitely an exception to the rule, since it isn't truly managing a real resource. It is not an example to be followed. We almost certainly could make improvements on its implementation now to be more "restful".
We've certainly made design decisions that we'd do differently the second time around, but they aren't justification for perpetuating similar decisions in newer designs.
0 -
Not mixing the soft delete into the general purpose PATCH request is an excellent idea. I agree with the others that PATCH /articles/{id}/status is probably what we should use.
We certainly do not want to put verbs into path names. Any such verby paths currently in the API have all been honestly and thoughtfully deliberated on by developers (mostly @Ryan and I). Exceptions to rules should be discussed between colleagues.
And on the topic of the "/edit" "verb". The word "edit" is not only a verb in the same way the word "post" is not only a verb. In addition to that, the usage of words amongst native English speakers can often take on new meaning and gain strength as Jargon in ways that may not be as obvious to non-native English speakers. In this case there is sometimes just an "it feels right" moment when naming a public endpoint, and I respectfully say that this is a decision that should be made by a native English speaker for our public API endpoints. In this respect, it's no different than having writers write our blog. We don't need to be this strict on any of our internal symbol names.
On the topic of @Adam Charron's need to get the record back in the PATCH request. This is technically not RESTful, but is kind of a limitation of REST that is a subject of much debate within the RESTful community. This kind of issue was one of the main driving forces behind GraphQL. For now, we should be looking to the front end team as the "customers" of the API and make sure they are getting what they need. I have a couple of suggestions on how we can somewhat ameliorate this issue, but for now we need to have the tradeoff.
0








