Everyone has different experiences but my view of the problem is that it's not so much down to developer laziness as it is technological ignorance and reactive direction from higher up - the people who actually fund and approve the work.
Outside of the bigger providers I don't think most APIs are designed with general consumption in mind. The impetus usually seems to be reactive - "we've got this idea for a mobile app using data from our legacy WMS/CRM/whatever, and it needs to be finished yesterday!"
So generally (not always but usually) what I've seen is whatever the underlying system supports and requires gets reflected into the API, and there's minimal proper engineering or sensible abstraction, partly because of time limits and also because the requirements are discovered mid-flight.
"Oh... Products have options in a separate entity? Oops. Uh, we'll add an options endpoint. Easy."
Several years later, the app is mildly successful, the business decides to whitelabel the API for some of their B2B clients, there's no budget for a redesign, "It works doesn't it? If it ain't broke don't fix it..."
Where possible I try to design and build APIs with some forethought and push for as much information on requirements and capabilities of existing systems beforehand but quite often the stakeholders don't really know and don't want to pay for a few weeks of in-depth analysis.
The endless facepalming of hearing the cycle of "Don't overengineer it" when I ask too many questions people can't answer, through to "Why didn't we pick up on this requirement" and having to listen to "we can all learn from this" when something fails because of some if/else statement buried in the legacy business logic, right back to "Don't overengineer it" when you propose the clean way of encapsulating that in the API, has left a permanent indent on my skull. So much leakage of black-box logic into the clients which bloats frontend logic and provides plenty of double handling and scope for unknown business logic to be missed, and no one in charge really gives a shit other than the developers and engineers, and that's fine because we can deal with it and it's their fault for not thinking of it anyway... they're the smart ones right?
You'd think this is just a problem with SMEs but some of the biggest national and multinationals I've worked with have a ton of horrid shit in their API designs. One industry-wide government-mandated reporting spec you'd post a SOAP envelope that wraps an undocumented XML document (they had an example document and the envelope was documented, but no doctype or anything to go off for the contents), a major real estate interchange format has you polling a REST API for an update flag, then loading and saving CSVs over FTP. Some godawful affiliate marketing API essentially just exposed their nth normal form relational database as an API. One government department just used HTSQL for their API using a python script to reflect some of their silos into PGSQL and called it a day (even though whether or not your interactions would work or made sense depended entirely on their internal application logic which wasn't represented or documented).
And too many projects where it turns out the promised APIs didn't even exist until the app was well into development. "Don't worry about it, that's their problem and gives us an excuse when the inevitable delays occur..."
No wonder we're hearing of massive breaches daily, from top to bottom the entire industry is barely held together by glue and sticky tape. It seems like an engineering problem but I think it goes much higher than that, it's a cross-industry failure in planning and management. Engineers are not given the time and budget to do the job that the data deserves. The market self corrects though, massive breaches have closed more than one business down for good.
It feels futile but I do support and encourage people to work towards just reasonable and well-documented endpoints, I doubt there's one true standard, but just some level above slapping it together at the last minute would be nice. I don't care if it's JSON or XML, or whether paths follow some semantics or another. As long as the relationships, types, and mandatory vs optional fields are explained clearly and succinctly, and the auth mechanism isn't a nightmare, and I can work with it.
Sorry for the rant this is just one area that triggers me. I've seen too much bad practice, and fighting against it for decades and still hearing the same lack of concern in the architecture phase has left me more than a little jaded.
Outside of the bigger providers I don't think most APIs are designed with general consumption in mind. The impetus usually seems to be reactive - "we've got this idea for a mobile app using data from our legacy WMS/CRM/whatever, and it needs to be finished yesterday!"
So generally (not always but usually) what I've seen is whatever the underlying system supports and requires gets reflected into the API, and there's minimal proper engineering or sensible abstraction, partly because of time limits and also because the requirements are discovered mid-flight.
"Oh... Products have options in a separate entity? Oops. Uh, we'll add an options endpoint. Easy."
Several years later, the app is mildly successful, the business decides to whitelabel the API for some of their B2B clients, there's no budget for a redesign, "It works doesn't it? If it ain't broke don't fix it..."
Where possible I try to design and build APIs with some forethought and push for as much information on requirements and capabilities of existing systems beforehand but quite often the stakeholders don't really know and don't want to pay for a few weeks of in-depth analysis.
The endless facepalming of hearing the cycle of "Don't overengineer it" when I ask too many questions people can't answer, through to "Why didn't we pick up on this requirement" and having to listen to "we can all learn from this" when something fails because of some if/else statement buried in the legacy business logic, right back to "Don't overengineer it" when you propose the clean way of encapsulating that in the API, has left a permanent indent on my skull. So much leakage of black-box logic into the clients which bloats frontend logic and provides plenty of double handling and scope for unknown business logic to be missed, and no one in charge really gives a shit other than the developers and engineers, and that's fine because we can deal with it and it's their fault for not thinking of it anyway... they're the smart ones right?
You'd think this is just a problem with SMEs but some of the biggest national and multinationals I've worked with have a ton of horrid shit in their API designs. One industry-wide government-mandated reporting spec you'd post a SOAP envelope that wraps an undocumented XML document (they had an example document and the envelope was documented, but no doctype or anything to go off for the contents), a major real estate interchange format has you polling a REST API for an update flag, then loading and saving CSVs over FTP. Some godawful affiliate marketing API essentially just exposed their nth normal form relational database as an API. One government department just used HTSQL for their API using a python script to reflect some of their silos into PGSQL and called it a day (even though whether or not your interactions would work or made sense depended entirely on their internal application logic which wasn't represented or documented).
And too many projects where it turns out the promised APIs didn't even exist until the app was well into development. "Don't worry about it, that's their problem and gives us an excuse when the inevitable delays occur..."
No wonder we're hearing of massive breaches daily, from top to bottom the entire industry is barely held together by glue and sticky tape. It seems like an engineering problem but I think it goes much higher than that, it's a cross-industry failure in planning and management. Engineers are not given the time and budget to do the job that the data deserves. The market self corrects though, massive breaches have closed more than one business down for good.
It feels futile but I do support and encourage people to work towards just reasonable and well-documented endpoints, I doubt there's one true standard, but just some level above slapping it together at the last minute would be nice. I don't care if it's JSON or XML, or whether paths follow some semantics or another. As long as the relationships, types, and mandatory vs optional fields are explained clearly and succinctly, and the auth mechanism isn't a nightmare, and I can work with it.
Sorry for the rant this is just one area that triggers me. I've seen too much bad practice, and fighting against it for decades and still hearing the same lack of concern in the architecture phase has left me more than a little jaded.