In terms of actually documenting something, examples are not documentation at all -- they are merely illustration. The annotations accompanying the examples (if any) are better documentation than the examples themselves.
Similarly, unix man pages desperately need examples. They are almost without fail written as an exhaustive reference for someone who already knows how to use the tool, which is a totally valid use case. But that means they're generally useless for someone trying to use a tool for the first time. Good documentation needs to have both.
Examples are best only for the beginner/occasional users. For more experience devs, you want regular docs, with full parameter list.
Case in point: requests. Google always drops me to the pages like Quickstart[0], which are full of examples. But they are useless for advanced users! Yes, even my limited brain can remember that you call "get" to issue HTTP GET. What other options does it take? Does it take a timeout? How do I pass in content-type? Does raise_for_status ignore 204?
Both have their merits, but if developer only has time for one, I'd go for proper doc.
It is order of magnitudes easier to understand behavior through examples than through a reference-like documentation. People suck at naming things, and then documenting them. Just today I spent the last few hours trying to figure out how to get something to work because a parameter only worked if another paramter had a specific value. There was too much magic to try and figure out the code, llms wrongfully told me the latest version doesnt support the parameter because it couldn't find any documentation.
Ultimately, creating a small example that I know should've worked and changing things in a way that I could narrow down the problem worked best.
The point to all this is that examples often encapsulate multiple things into a condensed format, that is extremely high fidelity (since it's typically a code example that runs and has output). Things like Api documentation in comparison typically cover a huge surface area, and have way less effort in maintaining them.
Also for the record, I absolutely don't trust libraries on behavior that isn't extremely obvious, explicit and so absolute that it's almost effortless to find out (I am talking first thing on README, entire sections showing that one thing, or definition through types). I just assume that things like behavior around status codes are going to change, and do my best in the calling code to mitigate issues as much as possible.
Examples are often crucially valuable, not just for beginners. Some of them can inject in five seconds an understanding equivalent to an hour of reading the very-objective docs and experimenting. I'm thinking of some git doc pages. But it applies just as well to single functions that are very simple conceptually, but hard to describe clearly and briefly and completely.
The developer has time for neither or both. Once the productivity barrier for one has been broken, the other is a tiny extra effort. In that case, they must provide both, except in the small minority of cases that are exceptionally self-describing, where one (usually docs) is sufficient.
Examples is for beginners. Beginners here refer to people that started to learn something, anything. But one you've got a conceptual understanding of the thing. You mostly need a full manual/reference.
A proper doc definitely has to provide the details. However starting with an example or two is a nice way to a give the users a quick overview. Even better when this example is interactive as for redis where you even have an example use case. See https://redis.io/docs/latest/commands/incr/
Of course we want a full list of public functions with all the info. But with just a list of functions it's often still not quite clear how you're supposed to setup and call them, unless you dig through the whole list and try to understand how they interact internally. A few short examples on "How do I init the library/framework so I can start using it", "How do I recover from a failure case", etc. makes the docs infinitely more accessible and useful!
>When jumping between projects, languages and frameworks, it takes a considerable amount of mental energy to restore the context and understand what is going on.
When feasible, I'd add repl environments. The ability to quickly modify and test the example can make up for a lot of the shortcomings documentation might have.
The Diátaxis framework [1] provides a nice suggestion of different types of documentation, and when to use them. None of these types
is "best", each serves a different purpose.
I think conceptually diataxis is brilliant. However, it is not trivial to implement. Every project needs a varying ratio of each component and stacking all forms in a single website in the same format is very ineffective. The ratio also evolves with community adoption and expertise level.
I really wish it was simply more actionable. Documentation is a hard problem, maybe we will figure out a better way one day. Until then docs will only be as good as the amount of time and expertise spent on them, which is usually not as high as it should be due to resource constraints.
This sort of thing really bugs me when I work with Unity and Unreal. Sometime the documentation for something is just so useless. A lot of the time when trying to understand Unreal nodes, the documentation page is just a picture of the node and the name of the node restated in slightly longer words, as if that helps anything. And so many times when I'm using Unity at £JOB, I just want to know how to use a function properly, and a short example would help so much. It's generally good but some pages just have nothing of value on them. If I could submit my own additions to the Unity docs pages, I would probably end up doing that
Obviously you want detailed documentation too, and examples are great for quickly grasping bigger concepts and how to integrate into other systems. In my experience the best solution are both, where examples all have hyperlinks to the actual documentation for everything involved in the example. React is a good example in my opinion; there is detailed documentation of the api, overviews of the concepts, as well as examples that are included where necessary and as part of a walkthrough.
Even better are the challenges at the end of their examples which force you to rethink what you just read.
Radical opinion: If the technical spec of a method cannot be intuited from the signature and a handful of canonical examples of usage, the method is probably trying to do too many things.
In particular, I don't want to have to learn half a dozen footguns because of a leaky abstraction.
No. Examples are there to show how to use this method, and other methods in conjunction.
You can intuit all you want from a method signature, and then you will fail to produce working code because you missed a config, or a preparation step, or don't understand how to process the results, or...
> because you missed a config, or a preparation step
If the specific values of parameters of your mutable state are material to the outcome, then the example is incomplete if they are not specified. Similarly, you wouldn't use `fib(x) = 3` as an example without specifying `x = 4` in your context.
> don't understand how to process the results
Not sure when this would be the case. Do you have an example in mind?
Right, one of the best out-of-box tooling decisions for Rust is that by default, and out of the box cargo will assume all your documentation examples are tests it should compile and execute.
This means so long as you run the tests before shipping you basically can't ship with examples you forgot to update after an API change because they either don't compile or don't work.
And you might think well, nobody would ship examples that don't even compile right? But on more than one occasions I've used a Microsoft C# library that comes with examples clearly hand made by their engineers (yes I'm that old that it couldn't be LLM output) without checking they work. Somebody has sketched out what ought to work, they haven't tested it, and a reviewer has seen that it looks right and OK'd it, but again, never tested that it works. This is extremely frustrating because the documentation is wrong but now what?
Personally I am much more tired of lengthy troves of API docs that, while fully specifying the system, do nothing to specify how one is actually supposed to use it in practice, such as the general flow of the program itself, which functions are relevant to what overall operations, in what order certain functions should be called relative to others, how one should handle errors, etc.
If I at least have some examples to get me started, then I can go digging through other documentation (or even just the code) to figure how to do more advanced things... but being unable to even get started is a much bigger frustration IMO.
One time my company paid $5k for a commercial x264 license, and when we asked for some documentation on how to get started with encoding some video frames using their C API, they simply responded "the code is the documentation." With no real examples to go on (at the time in the 2000s there actually wasn't much available online at all), this set me back a good two weeks that I felt was completely unnecessary and completely turned me off to interacting with them at all for the future.
We ended up switching to a hardware solution that was much easier, faster, well-documented, and way friendlier people to talk to.
examples often show the "spirit" of the invention it's intended usage if you like, which explains more what it does than name alone... also reason why I prefer wordy development notes than just a list of parameters it's useful for me to know whether it's what I want quicker than what I think I want based on name
As the above 8y old discussion and today’s blog post both say: yes, examples are but one part of a complete set of learning materials — practicals, theory, recipes, reference.
However, in a toss up, the greatest of these is the one you always want first: picking the thing up, tossing it about to get a feel of it, and getting your hands dirty!
For code, worked examples are particularly helpful because your dev environment can jump to the function definition, which typically has its own reference documentation.
The form the documentation takes depends on the audience.
If the audience is teammates, the _code_ itself is usually the best documentation.
Tests lie about edge cases. A test _might_ be demonstrating an edge case, or the suite might ignore large sections of edge cases.
Documentation is worse, it rarely documents edge cases. Country Y passes legislation and you need to implement it? Yeah, the docs aren't getting updated.
If your audience is a different team, API references (the headers/RPC message schema/javadoc/etc.) are better than written docs. Again, they are less likely to mislead you and are much more likely to work.
Unless, of course, the interfaces are telling fibs about what is allowed. Interfaces that are big buckets of parameters can do that.
Only if your audience is an external organization would documentation be a good primary reference. It meets them where they are, selling them on the capabilities of the software and bootstrapping them to the code.
In terms of actually documenting something, examples are not documentation at all -- they are merely illustration. The annotations accompanying the examples (if any) are better documentation than the examples themselves.
Similarly, unix man pages desperately need examples. They are almost without fail written as an exhaustive reference for someone who already knows how to use the tool, which is a totally valid use case. But that means they're generally useless for someone trying to use a tool for the first time. Good documentation needs to have both.
Checkout cheat.sh
I have this in a script: `curl cheat.sh/"$1"`
I'm always pleasantly surprised that many man pages do have examples. But, sure, they could be a lot more comprehensive.
I've developed the habit of checking the end of the page, because that's where the examples are, when there are any.
Totally agree but there is tldr for that.
I learned recently that tldr also has separate pages for subcommands.
tldr was created specifically to address this miscoming of the manpages. And it's recent enough to not ship with any distros.
It already exists - tldr. Use it side by side with man.
Examples are best only for the beginner/occasional users. For more experience devs, you want regular docs, with full parameter list.
Case in point: requests. Google always drops me to the pages like Quickstart[0], which are full of examples. But they are useless for advanced users! Yes, even my limited brain can remember that you call "get" to issue HTTP GET. What other options does it take? Does it take a timeout? How do I pass in content-type? Does raise_for_status ignore 204?
Both have their merits, but if developer only has time for one, I'd go for proper doc.
[0] https://requests.readthedocs.io/en/latest/user/quickstart/
It is order of magnitudes easier to understand behavior through examples than through a reference-like documentation. People suck at naming things, and then documenting them. Just today I spent the last few hours trying to figure out how to get something to work because a parameter only worked if another paramter had a specific value. There was too much magic to try and figure out the code, llms wrongfully told me the latest version doesnt support the parameter because it couldn't find any documentation.
Ultimately, creating a small example that I know should've worked and changing things in a way that I could narrow down the problem worked best.
The point to all this is that examples often encapsulate multiple things into a condensed format, that is extremely high fidelity (since it's typically a code example that runs and has output). Things like Api documentation in comparison typically cover a huge surface area, and have way less effort in maintaining them.
Also for the record, I absolutely don't trust libraries on behavior that isn't extremely obvious, explicit and so absolute that it's almost effortless to find out (I am talking first thing on README, entire sections showing that one thing, or definition through types). I just assume that things like behavior around status codes are going to change, and do my best in the calling code to mitigate issues as much as possible.
Examples are often crucially valuable, not just for beginners. Some of them can inject in five seconds an understanding equivalent to an hour of reading the very-objective docs and experimenting. I'm thinking of some git doc pages. But it applies just as well to single functions that are very simple conceptually, but hard to describe clearly and briefly and completely.
The developer has time for neither or both. Once the productivity barrier for one has been broken, the other is a tiny extra effort. In that case, they must provide both, except in the small minority of cases that are exceptionally self-describing, where one (usually docs) is sufficient.
Examples is for beginners. Beginners here refer to people that started to learn something, anything. But one you've got a conceptual understanding of the thing. You mostly need a full manual/reference.
I agree. Good documentation (nearly) fully specifies program behavior, and doing so with just examples is impractical or impossible.
A proper doc definitely has to provide the details. However starting with an example or two is a nice way to a give the users a quick overview. Even better when this example is interactive as for redis where you even have an example use case. See https://redis.io/docs/latest/commands/incr/
They're not mutually exclusive you know, you can have both example and a proper technical doc.
Took the words right outta my mouth!
Of course we want a full list of public functions with all the info. But with just a list of functions it's often still not quite clear how you're supposed to setup and call them, unless you dig through the whole list and try to understand how they interact internally. A few short examples on "How do I init the library/framework so I can start using it", "How do I recover from a failure case", etc. makes the docs infinitely more accessible and useful!
>occasional users
That's exactly what I'm pointing to in the post:
>When jumping between projects, languages and frameworks, it takes a considerable amount of mental energy to restore the context and understand what is going on.
[dead]
You need both and there are no ways around that.
Examples let you grasp immediately how to use the library and provide you good starting point for your integration.
Detailed explanation of all params and configurations allows you to solve more complex problems and understand the full capabilities of the tool.
I am miserable when any of the two kind of documentation is missing.
The only exceptions are very simple libraries, where the example tells you already everything there is to know.
Agreed, examples and reference docs.
When feasible, I'd add repl environments. The ability to quickly modify and test the example can make up for a lot of the shortcomings documentation might have.
The Diátaxis framework [1] provides a nice suggestion of different types of documentation, and when to use them. None of these types is "best", each serves a different purpose.
[1] https://diataxis.fr/
I think conceptually diataxis is brilliant. However, it is not trivial to implement. Every project needs a varying ratio of each component and stacking all forms in a single website in the same format is very ineffective. The ratio also evolves with community adoption and expertise level. I really wish it was simply more actionable. Documentation is a hard problem, maybe we will figure out a better way one day. Until then docs will only be as good as the amount of time and expertise spent on them, which is usually not as high as it should be due to resource constraints.
This sort of thing really bugs me when I work with Unity and Unreal. Sometime the documentation for something is just so useless. A lot of the time when trying to understand Unreal nodes, the documentation page is just a picture of the node and the name of the node restated in slightly longer words, as if that helps anything. And so many times when I'm using Unity at £JOB, I just want to know how to use a function properly, and a short example would help so much. It's generally good but some pages just have nothing of value on them. If I could submit my own additions to the Unity docs pages, I would probably end up doing that
Code examples can be executed as unit tests to prevent documentation regressions / bitrot in ways human language can't
Carefully chosen examples are best documentation. Badly chosen examples are a mediocre documentation.
It's a false dichotomy good documentation should have both - comprehensive reference and examples.
Obviously you want detailed documentation too, and examples are great for quickly grasping bigger concepts and how to integrate into other systems. In my experience the best solution are both, where examples all have hyperlinks to the actual documentation for everything involved in the example. React is a good example in my opinion; there is detailed documentation of the api, overviews of the concepts, as well as examples that are included where necessary and as part of a walkthrough.
Even better are the challenges at the end of their examples which force you to rethink what you just read.
Examples are the best `quick start` documentation. Comprehensive manuals/references are the best documemtation.
The best docs are the ones that satisfy user needs. https://passo.uno/seven-action-model/
Radical opinion: If the technical spec of a method cannot be intuited from the signature and a handful of canonical examples of usage, the method is probably trying to do too many things.
In particular, I don't want to have to learn half a dozen footguns because of a leaky abstraction.
No. Examples are there to show how to use this method, and other methods in conjunction.
You can intuit all you want from a method signature, and then you will fail to produce working code because you missed a config, or a preparation step, or don't understand how to process the results, or...
> because you missed a config, or a preparation step
If the specific values of parameters of your mutable state are material to the outcome, then the example is incomplete if they are not specified. Similarly, you wouldn't use `fib(x) = 3` as an example without specifying `x = 4` in your context.
> don't understand how to process the results
Not sure when this would be the case. Do you have an example in mind?
Even better if the documentation examples are part of the test suite!
Right, one of the best out-of-box tooling decisions for Rust is that by default, and out of the box cargo will assume all your documentation examples are tests it should compile and execute.
This means so long as you run the tests before shipping you basically can't ship with examples you forgot to update after an API change because they either don't compile or don't work.
And you might think well, nobody would ship examples that don't even compile right? But on more than one occasions I've used a Microsoft C# library that comes with examples clearly hand made by their engineers (yes I'm that old that it couldn't be LLM output) without checking they work. Somebody has sketched out what ought to work, they haven't tested it, and a reviewer has seen that it looks right and OK'd it, but again, never tested that it works. This is extremely frustrating because the documentation is wrong but now what?
They are not the “best.” They are helpful.
I am tired of rudimentary docs that only have examples.
Personally I am much more tired of lengthy troves of API docs that, while fully specifying the system, do nothing to specify how one is actually supposed to use it in practice, such as the general flow of the program itself, which functions are relevant to what overall operations, in what order certain functions should be called relative to others, how one should handle errors, etc.
If I at least have some examples to get me started, then I can go digging through other documentation (or even just the code) to figure how to do more advanced things... but being unable to even get started is a much bigger frustration IMO.
One time my company paid $5k for a commercial x264 license, and when we asked for some documentation on how to get started with encoding some video frames using their C API, they simply responded "the code is the documentation." With no real examples to go on (at the time in the 2000s there actually wasn't much available online at all), this set me back a good two weeks that I felt was completely unnecessary and completely turned me off to interacting with them at all for the future.
We ended up switching to a hardware solution that was much easier, faster, well-documented, and way friendlier people to talk to.
I agree that code is not documentation.
I agree that automatically generated API docs from code is nearly useless.
I agree that examples are important and useful.
But examples are not "the best documentation." Or I should say hyperbole is poor documentation of one's point of view.
examples often show the "spirit" of the invention it's intended usage if you like, which explains more what it does than name alone... also reason why I prefer wordy development notes than just a list of parameters it's useful for me to know whether it's what I want quicker than what I think I want based on name
tldr.sh is a project that had been doing this for man pages:
https://news.ycombinator.com/item?id=15779382
As the above 8y old discussion and today’s blog post both say: yes, examples are but one part of a complete set of learning materials — practicals, theory, recipes, reference.
However, in a toss up, the greatest of these is the one you always want first: picking the thing up, tossing it about to get a feel of it, and getting your hands dirty!
For code, worked examples are particularly helpful because your dev environment can jump to the function definition, which typically has its own reference documentation.
when possible (python, golang and many others) I just go and read the source..
also unit tests
I’d say the part of the unit test that is helpful is the example of using the code.
It has a pro over documented examples in that they are _guaranteed_ to be correct.
[dead]
This is why LLMs won.
Useless documentation means half-arsed is better.
The world of IT is broken, what sort of idiot gives Linux to their parents when as a trained developer man is so useless?
It's just excuse after excuse. Unit tests are documentation sort of garbage.
That's what's mind blowing about LLMs, IT devs are so bad LLMs are better. Hacker News comments also confirm this.
The form the documentation takes depends on the audience.
If the audience is teammates, the _code_ itself is usually the best documentation.
Tests lie about edge cases. A test _might_ be demonstrating an edge case, or the suite might ignore large sections of edge cases.
Documentation is worse, it rarely documents edge cases. Country Y passes legislation and you need to implement it? Yeah, the docs aren't getting updated.
If your audience is a different team, API references (the headers/RPC message schema/javadoc/etc.) are better than written docs. Again, they are less likely to mislead you and are much more likely to work.
Unless, of course, the interfaces are telling fibs about what is allowed. Interfaces that are big buckets of parameters can do that.
Only if your audience is an external organization would documentation be a good primary reference. It meets them where they are, selling them on the capabilities of the software and bootstrapping them to the code.
[dead]