Adding and Deleting Files

At first it might seem simple enough to just add another file header and file data block to the end of the current file list in flash. In the ideal situation, it would be. In reality, there are a lot of cases to consider. This section walks through the process of adding (with tfsadd() ) and deleting (with tfsunlink() ) files to/from TFS.

The tfsadd() Function

At a high level, TFS takes the following steps to add a file to the file system:

1. Does a file with the same name already exist in TFS?

   • If yes and if the existing file content is identical to the incoming content, then return success.

   • If yes but the new file content is different from the current file, the current file is marked stale, and the process of adding the new file continues.

   • If no, continue.

2. Run 32-bit CRC on the incoming data.

3. Is there enough room to store the file?

   • If yes, continue.

   • If no, defragment the TFS flash space. If after defragmentation there is enough space, then continue. If there is not enough space, return indicating that there is not enough flash space left.

4. Run a second 32-bit CRC on the incoming data. If first and second CRC values differ , then return an error (The reason for this two-pass CRC is described below.).

5. Transfer the data to flash memory, then build the new TFS file header, and copy that to flash.

6. If a file was previously marked stale , then mark it deleted. All done.

The next few listings (Listings7.4 to 7.13) are chunks of the tfsadd() function. This function deals with all of the complexities of adding a new file to TFS. The parameters name , info , and flags are character strings (see Listing 7.4). The variable name is the string to be used as the filename; the string info is used optionally to describe the file. The flags parameter is a string of ASCII characters that describe the attributes to be applied to the file. Note that info and flags can be NULL , in which case tfsadd() supplies defaults. The remaining arguments are src and size . The src argument is a pointer to the actual data that is to become the data portion of the file in TFS storage space, and size is the number of bytes to be considered for storage.

Listing 7.4: tfsadd() .

 int tfsadd(char *name, char *info, char *flags, uchar *src, int size) {     TDEV    *tdp;     TFILE   *fp, tf;     ulong   endoftfsflash, nextfileaddr, state_table_overhead;     ulong   crc_pass1, crc_pass2, bflags;     int     ftot, cleanupcount, err, stale, ssize;     if (!info) info = "";     if (!flags) flags = ""; 

 

Note 

The user can trace several functions in TFS. Set the tfsTrace variable to enable tracing (see Listing 7.5). The result of enabling tracing is that the call to tfsadd() is printed to the console for observation by the user.

Listing 7.5: The Trace Feature.

 if (tfsTrace > 0)         printf("tfsadd(%s,%s,%s,0x%lx,%d)\n",             name,info,flags,(ulong)src,size); 

 

Note 

TFS includes a perror -like" function that allows the user to convert a returned error code to some error message. Error checking can return specific codes that can later be passed back into another TFS API function for expansion to a more detailed description. The error checks done in Listing 7.6 demonstrate this feature.

The call to tfsFileIsOpened() near the end of Listing 7.6 is performed so that an executing script (scripts are executed directly out of flash) is not accidentally deleted by some unaware task.

Listing 7.6: Checking Operation Validity.

 /* Check for valid size and name: */     if ((size <= 0)  (!name))         return(TFSERR_BADARG);     /* If name or info field length is too long, abort now... */     if ((strlen(name) > TFSNAMESIZE)          ((info) && (strlen(info) > TFSINFOSIZE)))         return(TFSERR_NAMETOOBIG);     /* If the file is currently opened, then dont allow the add... */     if (tfsFileIsOpened(name))         return(TFSERR_FILEINUSE); 

 

Listing 7.7: Processing Attribute Flags.

 /* If incoming flags are illegal, abort now... */     if (*flags == 0) {         bflags = 0;     }     else {         err = tfsflagsatob(flags,&bflags);         if (err != TFS_OKAY)             return(err);     } 

 

The flags string passed in as an argument must be converted to a bit field ( unsigned long for this implementation) with certain bit settings representing specific characters in the flags string. (Accepting the flags input in character form makes it easy to allow a user to specify a set of flags at the CLI.) The call to tfsflagsatob() constructs the bit field and returns an error if the conversion fails.

Listing 7.8: Computing the Initial Checksum.

 /* Take snapshot of source crc. */     crc_pass1 = crc32(src, size); 

 

Before beginning a copy to flash, the code computes a 32-bit CRC on the incoming data. Actually, this CRC is computed multiple times so that tfsadd() can detect changing source data. This step is useful for two reasons:

• if the incoming data is changing, it is likely that the source address is incorrect.

• the incoming pointer should not be pointing to raw data in TFS space. If the incoming pointer is pointing to raw data and a defragmentation occurs, data in TFS is shifted.

Just before beginning the meat of the copy, the incoming name is checked against the possible list of multiple flash devices under TFS (see Listing 7.9). If the prefix of the incoming name matches the name of one of the devices, TFS assumes that the file is to be added to the associated flash device.

Listing 7.9: Identifying Flash Device.

 /* Establish the device that is to be used for the incoming file      * addition request... The device used depends on the prefix of      * the incoming file name. If the incoming prefix doesn't match      * any of the devices in the table, then place the file in the      * first device in the table (assumed to be the default).      */     for(tdp=tfsDeviceTbl;tdp->start != TFSEOT;tdp++) {         if (!strncmp(name,tdp->prefix,strlen(tdp->prefix)))             break;     }     if (tdp->start == TFSEOT)         tdp = tfsDeviceTbl; 

 

Listing 7.10: Finding the Last Header.

 tryagain:     fp = (TFILE *)tdp->start;     /* Find end of current storage: */     ftot = 0;     while (fp) {         if (fp->hdrsize == ERASED16)             break;         if (TFS_FILEEXISTS(fp)) {             ftot++;             if (fp->flags & TFS_NSTALE) {                 if (!strcmp(TFS_NAME(fp),name)) {                 /* If file of the same name exists AND it is identical to                  * the new file to be added, then return TFS_OKAY and be                  * done; otherwise, remove the old one and continue.                  * Dont do the comparison if src file is in-place-modify                  * because the source data is undefined.                  */                     if ((!(bflags & TFS_IPMOD)) &&                         (!tfscompare(fp,name,info,flags,src,size)))                         return(TFS_OKAY);                                      /* If a file of the same name exists but is different than                  * the new file, make the current file stale, then after                   * the new file is added we will delete the stale one.                  */                     stale = 1;                     err = tfsmakeStale(fp);                     if (err == TFS_OKAY)                         goto tryagain;                     else                         return(err);                 }             }         }         fp = nextfp(fp,tdp);     }     if (!fp)    /* If fp is 0, then nextfp() (above) detected corruption. */         return(TFSERR_CORRUPT); 

 

TFS searches the appropriate flash device looking for the last file stored on the device (so that it can append the new file to the end of the current list). This search consists of stepping through the linked list. During this search, TFS also verifies the sanity of each of the file headers. The call to nextfp() does some basic checks. At some point TFS finds the last header or returns TFS_CORRUPT, indicating that headers are incorrect. If the situation where the file to be added already exists (same name) in TFS, additional steps might be necessary.

• If the file exists and is identical in data, flags, and information field to the incoming file, the code immediately returns. Because the two file images are identical, there is no harm in skipping the copy to flash. In fact, doing so extends the component life by avoiding a flash erase/program cycle.

• If the file exists and is different, then the existing file is marked STALE . This file will eventually be replaced with the new file, but the code delays until the new file is successfully copied before actually deleting the current version.

• If the file does not already exist, the new file is added with none of the preceding complications.

When TFS finds the end of the current list of files, it must determine whether there is enough space to add the incoming file (see Listing 7.11). If enough space is available, the file is appended to the list. If there isnt enough space, the add routine initiates a defragmentation cycle.

When computing available file space, one must consider the overhead required for defragmentation. This overhead includes the spare sector, as well as the space needed for the defragmentation header and sector CRC table, both of which are built downward from the end of TFS space during defragmentation. After defragmentation is completed, TFS scans through the list a second time, finds the end point, and determines if there is now enough space. If yes, the add operation can continue; if not, the routine returns TFSERR_FLASHFULL .

Listing 7.11: Checking for Free Space.

 /* Calculate location of next file (on mod16 address). This will be       * initially used to see if we have enough space left in flash to store      * the current request; then, if yes, it will become part of the new      * files header.      */     nextfileaddr = ((ulong)(fp+1)) + size;     if (nextfileaddr & 0xf)         nextfileaddr = (nextfileaddr  0xf) + 1;          /* Make sure that the space is available for writing to flash...      * Remember that the end of useable flash space must take into      * account the fact that some space must be left over for the      * defragmentation state tables. Also, the total space needed for      * state tables cannot exceed the size of the sector that will contain      * those tables.       */     state_table_overhead = ((ftot+1) * DEFRAGHDRSIZ) +         (tdp->sectorcount * sizeof(long));     if (addrtosector((uchar *)(tdp->end),0,&ssize,0) < 0)         return(TFSERR_MEMFAIL);     if (state_table_overhead >= (ulong)ssize)         return(TFSERR_FLASHFULL);     endoftfsflash = (tdp->end + 1) - state_table_overhead;     if ((nextfileaddr >= endoftfsflash)          (!tfsSpaceErased((uchar *)fp,size+TFSHDRSIZ))) {         if (!cleanupcount) {             err = tfsautoclean(0,0,0,0,tdp,0,0);             if (err != TFS_OKAY) {                 printf("tfsadd autoclean failed: %s\n",                     (char *)tfsctrl(TFS_ERRMSG,err,0));                 return(err);             }             cleanupcount++;             goto tryagain;         }         else             return(TFSERR_FLASHFULL);     } 

 

Listing 7.12: Testing for Stable Data.

 /* Do another crc on the source data. If crc_pass1 != crc_pass2 then      * somehow the source is changing. This is typically caused by the fact      * that the source address is within TFS space that was automatically      * defragmented above. Since we are aborting the creation of the file,      * we must undo any stale file that may have been created above.      * No need to check source data if the source is in-place-modifiable.      */     if (!(bflags & TFS_IPMOD)) {         crc_pass2 = crc32(src,size);         if (crc_pass1 != crc_pass2) {             if (stale)                 tfsstalecheck(0);             return(TFSERR_FLAKEYSOURCE);         }     }     else         crc_pass2 = ERASED32; 

 

It is almost time to do the flash write operations. The next step is to perform a second CRC on the incoming data (see Listing 7.12). If the CRCs do not match, the routine returns an error indicating that the incoming data is not stable.

In preparation for the actual write, tfsadd() creates a new file header in the local structure tf (on the stack)(see Listing 7.13). After the new file header is created, the data portion of the incoming file is copied to flash and, finally, the file header is copied to flash. The file data is copied prior to the header so that the header is the very last flash write, making it easier to detect an interrupted write. After the new file is successfully copied, tfsadd() performs one more CRC to verify that the flash copy was successful. TFS can then delete any stale file that was left. The last step is a call to tfslog() , which, if enabled, keeps a log of the file operations that actually modify the flash memory.

Listing 7.13: Creating the Header.

 memset((char *)&tf,0,TFSHDRSIZ);     /* Copy name and info data to header. */     strcpy(tf.name, name);     strcpy(tf.info, info);     tf.hdrsize = TFSHDRSIZ;     tf.hdrvrsn = TFSHDRVERSION;     tf.filsize = size;     tf.flags = bflags;     tf.flags = (TFS_ACTIVE  TFS_NSTALE);     tf.filcrc = crc_pass2;     tf.modtime = tfsGetLtime();     tf.next = 0;     tf.hdrcrc = 0;     tf.hdrcrc = crc32((uchar *)&tf,TFSHDRSIZ);     tf.next = (TFILE *)nextfileaddr;     /* Now copy the file and header to flash.      * Note1: the header is copied AFTER the file has been      * successfully copied. If the header was written successfully,      * then the data write failed, the header would be incorrectly      * pointing to an invalid file. To avoid this, simply write the      * data first.      * Note2: if the file is in-place-modifiable, then there is no      * file data to be written to the flash. It will be left as all FFs      * so that the flash can be modified by tfsipmod() later.      */     /* Write the file to flash if not TFS_IPMOD: */     if (!(tf.flags & TFS_IPMOD)) {         if (tfsflashwrite((ulong *)(fp+1),(ulong *)src,size) == -1)             return(TFSERR_FLASHFAILURE);     }     /* Write the file header to flash: */     if (tfsflashwrite((ulong *)fp,(ulong *)(&tf),TFSHDRSIZ) == -1)         return(TFSERR_FLASHFAILURE);     /* Double check the CRC now that it is in flash. */     if (!(tf.flags & TFS_IPMOD)) {         if (crc32((uchar *)(fp+1), size) != tf.filcrc)             return(TFSERR_BADCRC);     }     /* If the add was a file that previously existed, then the stale flag      * will be set and the old file needs to be deleted...      */     if (stale) {         err = _tfsunlink(name);         if (err != TFS_OKAY)             printf("%s: %s\n",name,tfserrmsg(err));     }     tfslog(TFSLOG_ADD,name);     return(TFS_OKAY); } 

 

The tfsunlink() Function

The file delete function, tfsunlink() (see Listing 7.14) does not require as much error checking as tfsadd() because the only incoming argument is the name of the file to be deleted. As did tfsadd() , the unlink function implements the trace mechanism and checks to see if the file is currently open . The delete code then retrieves a pointer to the files by calling tfsstat() . If tfsstat() returns NULL , tfsunlink() assumes there is no file in TFS with the incoming name and returns the appropriate error code. If tfsstat() returns a pointer, then that file headers TFS_ACTIVE bit is cleared to indicate that the file has been deleted. The final step is to call tfslog() to record the flash transaction.

Listing 7.14: tfsunlink() .

 int tfsunlink(char *name) {     TFILE *fp;     ulong flags_marked_deleted;     if (tfsTrace > 0)         printf("_tfsunlink(%s)\n",name);     /* If the file is currently opened, then dont allow the deletion... */     if (tfsFileIsOpened(name))         return(TFSERR_FILEINUSE);     fp = tfsstat(name);     if (!fp)         return(TFSERR_NOFILE);     if (TFS_USRLVL(fp) > getUsrLvl())         return(TFSERR_USERDENIED);     flags_marked_deleted = fp->flags & ~TFS_ACTIVE;     if (tfsflashwrite((ulong *)&fp->flags,         &flags_marked_deleted,sizeof(long)) < 0) {         return(TFSERR_FLASHFAILURE);     }     tfslog(TFSLOG_DEL,name);     return (TFS_OKAY); }