Documentation

---

I am not a fan of creating documentation simply for the sake of documenting. However, with an animal this big and this complex that requires this much attention, it is always a good idea to take good notes when you are dealing with a patching session or troubleshooting issues or just with getting answers to questions that you need to have answered.

Once you have taken the notes, transcribe them into an electronic format and put them in a central location (one that routinely gets backed up) so that you can easily find them later and remember what you did in a certain situation (when you are faced with a frighteningly similar situation later) and so that someone else who might run into a similar situation will be able to look at your document and have some idea where to start looking if they happen to be the one called for a problem that they have never seen. If nothing else, it will likely be a place for them to start instead of calling you or paging you several times simply because they do not know where to start or what might have already been tried.

No matter if you are dealing with Oracle Support, answers to questions from colleagues off of a listserv, or, if you're lucky, from someone inhouse helping you with the installation, whatever they tell you, take notes, and keep them. Something someone said might be one of the most useful pieces of information later, when everyone is gone and all you can remember is that you heard something sometime, but cannot remember who or even why you asked. If you are one of the key people on the implementation team and you know what you are doing, you are ahead of the game. If you do not know what you are doing or only have a vague idea of what really is going on, you will probably be very tired by the time your go live date arrives and so much will have happened that you will not be able to completely remember what happened early on. If there was a patch that broke badly in one environment and you fixed it, you should find a way to remember what broke, what you did to fix it, and where you looked to find the answer. It seems a lot like documentation, but there is a good chance that this documentation is for you and you will be much better off if you have it. And if it is not for you, but someone else that you work with, passing on well documented fixes will mean less frantic phone calls later when maintenance of the system is under the watchful eye of someone else.

This advice is not just for the technical people. Functional people on the team are probably more likely to take notes. Make sure that you do. Again, if you have a company in to help you through the implementation and with training of key people, you have this help at hand. They are captive and they will likely show you tricks for taking shortcuts, different ways of doing the same thing, or give you quick little hints that you will find useful after they are gone. The tips are great unless you cannot remember them when you are on your own and have to figure out how to do the very same thing. Do not let anyone say, "I'll fix it, wait a second." If they do the fix, you will not catch everything because it will happen too fast. Make them let you do the work with them prompting you. You will remember it much better that way.

In your notes, keep track of your contacts. Get names, addresses, and e-mail addresses of the people that you have found most helpful. Pick their brain later if you have questions that you think they can answer. Do not be a pain in the neck; they will tire of endless questions that you could find easily in a manual, but they will be glad to answer questions if you think ahead about what you are asking. Many times you will find someone who knows a lot about one particular part of the system. You may find someone who is an expert at printers or at Concurrent Managers. Make note of that. Later you may have a question on that very subject and while you are searching manuals and Web sites for answers to your question, take five minutes and send the person an e-mail. They may shed light not only on your question, but on surrounding questions as well.

Notes will help you answer questions going forward, too. If you know that you installed the D driver for the 11.5.8 upgrade patch and it took 13 hours to run on your upgrade of the Vis instance, you can be fairly sure that it will not take 5 hours on Prod, even if you have a faster machine or put more workers to work on it. When you plan for your next upgrade or major, regularly scheduled patching session, you will have a better idea that, if a 100 MB patch took 2 hours before, and a 75 MB patch took an 1¹/₂ hours before, then that 95 MB patch that you are looking at installing next time will likely take somewhere either around or between those two times. It is easier to justify a 3-day outage for patching or upgrade if you have the historical statistics to back you up. Notes will give you those statistics.

Along the same lines, keep logs of what patches you installed; when you installed; which instance they have been applied in; how long it took for each driver; any prepatch, midpatch, or postpatch steps that you had to do; the times that each of the additional steps took; and any problems (with resolutions) you had. Keep this information together in tables in the database or in an Excel spreadsheet that you maintain every time you patch, clone, or upgrade. This way you can quickly look back at what you did where and when and learn from what broke in another patch. If you have seen an error before, you might save yourself the time and aggravation of an iTAR. It will also help you with planning going forward.

Make the documentation fit you and fit your situation. Every situation is different. Every company is different. Every implementation is different. This is one of the wonderfully terrible things about Oracle Financials as it is in this iteration of its life; it is so flexible that there is rarely one absolutely definitive answer. I can give you some ideas that I have seen tried and some that might work in your situation. They are at least a place to start.

Create and Maintain a Patching Log

Yes, the newer maintenance releases of 11i come with some really neat new tables and interfaces that allow you to see what you have installed in your environment. It will let you see what minipack level you are on, what bugs were taken care of by patches that you have installed, and will even allow you easy access to how many times you started each driver in a particular patch. It will not tell you what you did to resolve the issues, who you talked to (especially if you have worked with a particularly good analyst), it will not track iTAR information that might have been created for an issue that came up, and it will not currently maintain information on exactly how long it took for each driver to run, and the differences in runtime between your different environments. Any of this information you may find interesting or helpful or critical at some point. If you maintain this information in a central location, it will be easier for you or others to go back and see what went on when.

Relevant iTAR information may be particularly important to keep if you want to track that kind of information, because Metalink only keeps a certain amount of iTARs listed online and then only for a certain period of time. After that, you will at least have to know the iTAR number and then you may not be able to retrieve the desired information.

Use Self-Documenting Log Filenames

There are many default log names that you will encounter in your illustrious career. When you run ADADMIN, there is a default log name; when you run adctrl, there is a default log name; and when you run ADPATCH, there is a default log name. They are short, fairly concise, and have a measure of usefulness. They are not very descriptive and they cannot easily be used for self-documentation. Personally, I thank my lucky stars for the ability to use long filenames. I am not a big fan of typing. I was a COBOL programmer and got a lot of practice with typing. I would much rather use three keystrokes over five if I can. But meaningful, self-documenting filenames are the best thing I have found yet to quickly tell me exactly what has happened. This is especially true if you are not the only one running utilities.

When I run ADPATCH, I make the log name as descriptive as possible. Usually the for mat of the name is something like <sid>_<driver>_<date and time started>_<initials of analyst installing the patch>.log. If it is a restart, you can tell what order you did them in and how many problems you had with each driver. You can infer (from the combination of the filename and the time stamp) how long that driver took to run. If you make the name more descriptive, you can tell why you restarted it or what you fixed in the meantime.

I usually follow similar logic when running ADADMIN. If I am running it to fix a problem, I put the problem in the filename (vis_form_abcdfmx_recompile_0428031224_ajw, I fixed form abcd on the 28^th of April at 12:24 P.M.). If it is a step in the installation of a patch, I put the patch name and where it falls in the patch in the filename (vis_post_ d2934565_recompile-flex-fields_0428031224_ajw). Granted, when you have been running patches for 15 or 18 hours straight, the extra typing gets to be a pain, but when you have been running patches for 15 or 18 hours straight, you want to remember later why you did what you did and when you did it.

There are times and places where you cannot change the log names. If a patch's C driver relinks files, the relink and rebase logs are default names. If you are running the D driver, you will have a worker log for every worker that you have running in the driver. When each driver of the patch is done running, rename these logs to something meaningful. This will help you in several ways. You will know what files get associated together later and you will have files that are not several hundred megabytes in size to wade through.

If you take the defaults, every time the utility that creates that file runs, it appends to the file. This is true for ADADMIN, ADPATCH, adrelink, and all of the other utilities. A single log is not bad if you really want to keep them all together, but it is not the greatest if you need to hunt for an error in a worker log and the log takes 10 minutes to load into your word processor.

Use the Tools That You Are Comfortable With

You can maintain your electronic version in a file system on whatever platform you choose in text file format so it is easily readable from any text editor. Or you can store it in a Windows-based file server in Word, Excel or Access format, text format, or in HTML/RTF format so it is browser friendly and you can link it and index it through the intranet. I have even seen much of the information on patching and problems with resolutions stored in the database. You can create tables in a users tablespace that store your information relevant to that environment. When that environment gets cloned, the information in those tables goes with it. You can create a documentation instance, independent of the central Applications databases in which you can store information on tuning measures you have tried with their results in each environment and idiosyncrasies that have popped up that you want to maintain ongoing that cloning one environment to another may overlay. Even if you only maintain the information in varchar2 columns, 4000 characters is a considerable amount to be able to store for notes. If you need more than 4000 characters, Clobs should be big enough or even work with bfiles to store the information. If you have Forms, you can even make a quick interface into these kinds of tables to make entering the information easier.

---