Practical self-hosting · intermediate
22 · The plug fits, but does the conversation?
Test API compatibility as observable behavior.
What you can take away
- Build a contract test matrix for a model client.
- Use fake responses or an isolated test endpoint
Your trail through this idea
Checking saved progress…
Ian’s LocalAI/LocalAGI linkage questions expose a common trap: two systems can advertise compatible APIs while disagreeing on the features your application actually uses.
Prerequisite: complete the preceding lesson or bring its worksheet.
Understand the system
Ollama documents support for a subset of the OpenAI API. The word 'compatible' should therefore trigger a feature checklist, not a blanket assumption. Authentication, model names, message roles, streaming, structured output, tool calls, error responses, and cancellation are separate behaviors. Test the subset your client requires against the exact server version.
A gateway can normalize some differences, but every normalization is another behavior to validate. Silent fallback is particularly dangerous when it changes privacy, cost, or tool permissions. A rejected request can be safer than an undisclosed route to a different provider.
Work through the case
Original contract matrix: a basic text request succeeds; a streaming response ends correctly; an invalid model returns a documented error; a tool call preserves its identifier; an interrupted request releases resources. Give each row an expected result and a captured sanitized observation. Do not mark the whole API compatible because the first row passed.
Make something useful
Try this
Original exercise
Use fake responses or an isolated test endpoint. Write five tests covering success, malformed input, unsupported capability, timeout, and cancellation. Keep prompts synthetic and cap output. If running real requests, record server version and the model identifier without exposing endpoint addresses or keys. Revert any temporary gateway configuration after the test.
Runnable contract lab
This Python 3 lab uses fake responses, no network, credentials or model server. Save it as lesson_contract.py and run python3 lesson_contract.py. Python's unittest documentation explains the assertions. The fake contract is deliberately smaller than any real inference API.
import unittest
def read_text(payload):
if not isinstance(payload, dict):
raise ValueError("response must be an object")
choices = payload.get("choices")
if not isinstance(choices, list) or len(choices) != 1:
raise ValueError("exactly one choice required")
choice = choices[0]
if not isinstance(choice, dict):
raise ValueError("choice must be an object")
message = choice.get("message")
if not isinstance(message, dict):
raise ValueError("message required")
text = message.get("content")
if not isinstance(text, str):
raise ValueError("text content required")
return text
class ContractTests(unittest.TestCase):
def test_valid_text(self):
self.assertEqual(read_text({"choices": [
{"message": {"content": "Synthetic answer"}}]}), "Synthetic answer")
def test_missing_choice(self):
with self.assertRaises(ValueError):
read_text({"choices": []})
def test_nontext_content(self):
with self.assertRaises(ValueError):
read_text({"choices": [{"message": {"content": None}}]})
if __name__ == "__main__":
unittest.main()
Expect three passing tests and OK. Change the valid fixture's content to a list and observe the failure. That is the point: a response can be valid JSON while violating the client's required contract. Restore the fixture afterward. This does not test streaming, authentication, tool calls or actual server compatibility; those need separate fixtures and integration tests. Remove the practice script for cleanup.
Check your reasoning
A quick check · for your own learning
What is an honest API compatibility claim?
A self-study prompt. You can retry; this is not an exam.
Keep the result and follow the sources
Success is a compatibility statement with a scope: 'the tested behaviors passed on this version.' That is more useful than a universal compatibility badge.
Sources and scope
- Primary documentation or archival source, reviewed 14 November 2026. Additional direct links appear beside the claims they support.
- Editorial basis: Ian's historical self-hosting questions and current repository inspection informed the case selection. Historical requests do not establish current deployment. Private chat records, credentials and host details are excluded.
- Examples and exercises are original educational synthesis. External sources support the stated technical or historical context, not every hypothetical scenario.