Tips for doing "emergency documentation" of complex embedded project before leaving? [closed]

Question

I am looking for advice for doing "emergency documentation" for a large and complex software module in the technical/embedded domain, that I worked on for several years.

I am going to leave the company within weeks, due to some developments which don't leave me another choice. The domain is more quality-conscious than the proverbial Internet startup. Projects often take years to complete. However, software is only part of the project, and due to severe time and cost pressures, managers had a tendency to focus on urgent issues while ignoring more long-term quality issues, which technical documentation is a part of. The idea was probably to produce some documentation at the very end of the project.

Now, the project manager realized that they will need an estimated six to eight months for getting the next developer (which has yet to be found) up to speed with the software. It will also be difficult to hire a sufficiently competent developer in a short time, given that the salary is below average.

Now, managers want me to "just document everything", which is hardly realistic given the size of the project. I also have the impression that they do not have a clear picture of what they need now. Only one of the involved persons has a strong software background.

What exists are some requirement specifications which are however incomplete. The code itself is already commented at a reasonable level. It is not deliberately tricky but some stuff needs specialist knowledge and/or a lot of experience. I already documented many technical decisions and considerations by email, writing weekly reports on strategy and technical difficulties. There is also already some good API documentation that I somehow managed to sneak in. What is needed is essentially some technical documentation which allows the next developer, assuming he is competent, to understand the larger picture so that he can relatively quickly continue working on it. I will need to assume that the reader of the documentation is somewhat proficient in C++ and is able to familiarize himself with any extra concepts he might need.

What do you recommend? In your experience, which aspects are most important and / or helpful? Would you recommend to use some kind of Wiki for documentation?

Answer 1

The temptation for this kind of thing is to try to cover everything in detail which isn't practical or necessary.

I'd advise doing a mind map first to get it clear in your own mind all the aspects that could need covering. Once you have this, you should be able to see at a glance what the critical areas are in order for the next developer to hit the ground running. The mind map will also give a helicopter view to the person coming in. This is crucial since it gives an idea of scope as well as introducing certain terminology that may be pertinent should they need to ask a SME for help.

You'll need some sort of colour coding to make it clear which areas are critical to know, which bits are otherwise useful and which bits can be skimmed or ignored entirely (code/processes that never change).

Now you have your critical set - document the hell out of these and then move onto the next priority until you run out of time. I say "run out of time" rather than done because - even with the best will in the world there is going to be stuff you forget or even things that haven't cropped up yet.

As an aside, the "document everything" approach is lazy management - pure and simple. If something is missed, they can just simply say it should have been covered.

Answer 2

First of all, it's very professional from you to try to find the best way to document your work given that it will soon no longer be your concern.

What to put in an urgency documentation ?

Unfortunately, it's a tough question; there are many "urgency documentation" out there that are useless, despite having been prepared with good intention. Documenting in a usefull way takes time.

I have no magical solution to propose, but only some thoughts to drive your effort:

• First, after you've left, your code will still be there. No need to document what a professional developper can easily find out from the code. Instead of making lengthy, cumbersome documentation of the code that will soon be obsolete, better improve some comments of the less obvious parts (if the code is not already self-explanatory).
• Second, the rest of the company still will be there after you left. So no need to complete requirements, except those that have a non-trivial effect on core effects of your design. If maintenance on your work is to be done, requirements will anyway have to be cross-checked with the stakeholders who own them.
• Third, you have certainly worked with your peers about the interactions of their part with your parts. So they should know how the interface with your code works. If you have some time left to document this external interface, do it. But if you don't, do not woryy to much: the core knowledge should already be shared.
• Well, what remains then ? The most difficult to find out for your successor, will be the internal design of your components, and especially the dynamics of the software, i.e. the interaction between the different classes. So this is something to start with: identify the responsibility of the classes, find the main interactions involving several classes, and describe how these interactions do work. Keep it simple: no details (can be found out in the code), but the big picture;

You could also consider using javadoc or doxygen. These generate class based documentation. Personally I find these useful only if provided in complement of the design documentation that gives the big picture. Be aware also not to use these tools to produce a full depth documentation with all the details. Details can be verified in the code by your successor. So if you use this kind of documentation, focus on the responsibilities/purpose of the classes and the members, to accelerate the learning by your successor

Is there a better approach ?

There absolutely is no guarantee that ex-post documentation will be understood end effective. You could therefore propose another approach:

1. write a tiny hand-over documentation (main packages, where to find the source code, main components with their purpose).
2. propose a knowledge transfer with your peers: share the functionality to cover between your team-mates and organise peer-review sessions where you can transfer from person-to-person the knowledge (most effective knowledge transfers are always from mind to mind).
3. Ask your peers in these sessions what they think should absolutely be documented. This will enable you to only cwrite useful documentation. Ideally, they could then even read your document before you leave and confirm if it's ok or not (e.g.avoid ambiguities).

In this way, the knowledge will stay in the team, even if not in a central place. Your successor can then learn the other way round, benefiting from peer-review by your team-mates. In addition there'll be some really useful documentation left.

• Inconvenience: it takes some resources since your team-mates have to give some time. And twice: when you leave, and again when your successor comes-in. So your short-term oriented managers might not like it.
• Advantages (out-weighting the inconveniences IMHO):
  • improved team-cohesion
  • more resilient team
  • effective (and verifiable) knowledge transfer
  • practice worth introducing anyhow
  • in case of urgent changes required in your part of the code in the 6 to 8 month between your departure and the arrival of the successor, the team will not be stuck, but could - thanks to the shared understanding - cope with it.

Alone in view of the last advantage, as a project manager, I'd really prefer this option !

Answer 3

Whatever the "development of things", a company you have worked for several years is no one-night stand (although I understand mileages may vary, sometimes a lot). You know a lot about whatever it is you have been working on and your experience is very valuable. Regardless of the terms at which you are with your manager(s), this is an irrefutable fact and you can be sure that it is one they know and understand quite well.

You will probably need to understand in advance that even after you leave, you will very likely still be in contact, occasionally, with the company, be it with the project manager, the new developer, or whomever gets dragged in for this, most likely to give bits of assistance and guidance here and there. It's your work after all. The best thing to do is to be prepared to provide it with as little "friction" as possible. If you don't really mind the company at all, at least you could do something about it for the next poor developer who will probably end up being either underpaid or "under-experienced".

Urgency documentation is very much likely a desperation move on behalf of your manager. Given that you have ended up dedicating a question here, you probably do want to do your best. If you already have comments in place and documented technical decisions and all this stuff you have mentioned, you are probably no different than a lot companies and how the job is done usually. Writing documentation is not part of your job (especially when time's on your head like an elephant), it's part of your ethical responsibilities and, as a result, you only do it well if you feel like it. It's also a thing we screw up a lot. A wiki can be great, but a wiki ends up being a "job" in and of itself.

I am afraid that trying to force-document things will simply end up being an extensive loss of time. A sad part of reality is that most developers do their absolute best to skip the documentation, they simply read code (as hard as already is, of course), trying to understand everything from there. This is already a bad practice in and of itself, do not make it more difficult by creating documentation that developers will be enticed to skip (documentation for clients and users is a completely different animal of course).

A suggestion would be to devote a bit of time to make whatever documentation you are going to compose accessible to you when you are not going to be in the company, so you can keep adding to it occasionally (so yes, a Wiki would be great in that respect). However, my primary suggestion would be to try to make yourself available, even if sparsely. There is nothing the next developer would want more than to be able to access you as painlessly as possible, if only for a couple of times. Based on most people I know in similar shoes, you are going to be getting "some" calls every now and then, at least if you have been a significant person for the project. Knowing that you are necessary, maybe you could make a "different" deal, one of occasional potential assistance and guidance (and, of course, set your "price" on that).

Managers request documentation out of a wish to minimize lost time, but they seem to miss the ways developers read and understand other developers' code (which will be the "lost" time in that case). They often do not understand what a developer API is. If your API is fairly documented, there is little else you can do to help other developers while absent. You know what type of input would help you learn from someone else whatever is needed if you were to take over their work, so you can do the same thing for them.

A small issue tracking infrastructure, just between developers, would be great. As great as it is for customers, the same (and maybe greater) it is for developers. Also, no amount of documentation can match you, being next to the developer and communicating directly your thoughts, decisions etc. If you can make a few visits after a new developer is in place, to give some direct guidance for a head start, the new developer will appreciate it a lot.

Is any of this possible, likely, or even desirable from/for you? If not, the simple answer to your question is to document whatever you would like to know if you were to take over your own codebase as another person. High-level stuff, probably. I know it's vague, but only you know what would be important in terms of the domain, in order to be able to maintain the codebase. If yes, however, then maybe you could try to also make a small deal out of it.

Answer 4

Whenever I read and try to reason my way through a foreign codebase, the hardest time I have (how hard depends, of course, on how well structured it is) is to learn, understand or guess what is relevant where. I may see methods with great and clear descriptions, but I want to find out where they are used and why, that is, the interactions and relations between elements. In my opinion, this is going to be the hardest part to explain and this is quite hard to achieve through text. This is why my previous answer was a bit idealistic.

The one largest problem with code is that it cannot talk! So why don't you talk instead? An interesting way, in my opinion, in which you can capture that much information, along with complicated combined directions is to make your documentation somewhat "interactive". It is clear that you are going to be completely out of reach, but you could try "leaving a part of yourself" there...you can try to screen capture a series of videos running through your codebase and explaining various pieces of the code along with whatever you consider important the "higher scheme of things".

You can break your videos in series, making it modular and of increasing complexity. You may think that whatever you would "narrate" in the video could just as well be written, but the most invaluable part will be the transitions, from method to method, class to class, file to file, sequentially documenting what you the programmer, had in mind when you were writing. Also, you can more easily directly point to complications, pieces of not-so-great organization, things that are hard to put together just by reading, all kinds of funky implementations "comprising" the technical debt.

Think about it... you can produce a lot more documentation (with much more condensed meanings than plain text can capture) in a much shorter time frame, so new developer(s) find(s) it easier to digest, get(s) acquainted faster, can review it as many times as necessary, everybody is happy.

I know it may sound a weird idea and might appear very informal and awkward, but "emergency documentation" sometimes comes in "emergency formats".

Answer 5

I would go with a top-down approach.

Assuming your code is well-written and well-documented the nitty gritty stuff is done.

Functional specification is not your job - so urge the management to update it.

Give an overview of your code, starting from a high level architecture and design decisions and branching down in order of importance. Best would be cross-referenced with code. You can even do it doxygen style where large explanations are at the beginning of the code file (to be included in the generated documentation). To speed up the process wherever possible link to external materials which you believe are good enough (no need to explain /sys/class/ stuff yourself) or even state that it's outside the scope/they need to search for it themselves.

Also, absolutely, include stuff about your hardware architecture. Add a block diagram of your hardware or a simplified schematic, link SoC reference manuals, whatever is relevant.

Lust, but not least, class and call diagrams. This can be autogenerated using doxygen. If it helped me with U-Boot it will help elsewhere too. If you are not keen on keeping autogenerated files at least include a Doxyfile tuned for your project.