From gleixner at infradead.org Fri Apr 1 02:21:48 2005 From: gleixner at infradead.org (gleixner@infradead.org) Date: Mon May 15 21:51:44 2006 Subject: mtd/drivers/mtd/nand nand_base.c,1.137,1.138 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv9108 Modified Files: nand_base.c Log Message: Use oobfree to calculate the number of oob bytes available for fs usage Index: nand_base.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/nand_base.c,v retrieving revision 1.137 retrieving revision 1.138 diff -u -r1.137 -r1.138 --- nand_base.c 24 Mar 2005 14:33:22 -0000 1.137 +++ nand_base.c 1 Apr 2005 07:21:44 -0000 1.138 @@ -2512,12 +2512,9 @@ /* The number of bytes available for the filesystem to place fs dependend * oob data */ - if (this->options & NAND_BUSWIDTH_16) { - mtd->oobavail = mtd->oobsize - (this->autooob->eccbytes + 2); - if (this->autooob->eccbytes & 0x01) - mtd->oobavail--; - } else - mtd->oobavail = mtd->oobsize - (this->autooob->eccbytes + 1); + mtd->oobavail = 0; + for (i = 0; this->autooob->oobfree[i][1]; i++) + mtd->oobavail += this->autooob->oobfree[i][1]; /* * check ECC mode, default to software From nico at infradead.org Fri Apr 1 10:36:15 2005 From: nico at infradead.org (Nicolas Pitre) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd mtdchar.c,1.69,1.70 Message-ID: Update of /home/cvs/mtd/drivers/mtd In directory phoenix.infradead.org:/tmp/cvs-serv10779/drivers/mtd Modified Files: mtdchar.c Log Message: reset file position when switching OTP mode Index: mtdchar.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/mtdchar.c,v retrieving revision 1.69 retrieving revision 1.70 diff -u -r1.69 -r1.70 --- mtdchar.c 17 Mar 2005 19:56:08 -0000 1.69 +++ mtdchar.c 1 Apr 2005 15:36:11 -0000 1.70 @@ -583,6 +583,7 @@ case MTD_OTP_OFF: break; } + file->f_pos = 0; break; } From nico at infradead.org Fri Apr 1 11:36:29 2005 From: nico at infradead.org (Nicolas Pitre) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/chips cfi_cmdset_0001.c,1.174,1.175 Message-ID: Update of /home/cvs/mtd/drivers/mtd/chips In directory phoenix.infradead.org:/tmp/cvs-serv10971/drivers/mtd/chips Modified Files: cfi_cmdset_0001.c Log Message: Index: cfi_cmdset_0001.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/chips/cfi_cmdset_0001.c,v retrieving revision 1.174 retrieving revision 1.175 diff -u -r1.174 -r1.175 --- cfi_cmdset_0001.c 1 Apr 2005 01:59:52 -0000 1.174 +++ cfi_cmdset_0001.c 1 Apr 2005 16:36:25 -0000 1.175 @@ -2062,8 +2062,20 @@ /* we need real chips here not virtual ones */ devsize = (1 << cfi->cfiq->DevSize) * cfi->interleave; chip_step = devsize >> cfi->chipshift; + chip_num = 0; - for (chip_num = 0; chip_num < cfi->numchips; chip_num += chip_step) { + /* Some chips have OTP located in the _top_ partition only. + For example: Intel 28F256L18T (T means top-parameter device) */ + if (cfi->mfr == MANUFACTURER_INTEL) { + switch (cfi->id) { + case 0x880b: + case 0x880c: + case 0x880d: + chip_num = chip_step - 1; + } + } + + for ( ; chip_num < cfi->numchips; chip_num += chip_step) { chip = &cfi->chips[chip_num]; otp = (struct cfi_intelext_otpinfo *)&extp->extra[0]; From dbrown at infradead.org Mon Apr 4 14:02:26 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand nand_base.c,1.138,1.139 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv7699 Modified Files: nand_base.c Log Message: Version 1.137 broke nand_read_ecc clients who pass NULL oobsel. Fixed. Index: nand_base.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/nand_base.c,v retrieving revision 1.138 retrieving revision 1.139 diff -u -r1.138 -r1.139 --- nand_base.c 1 Apr 2005 07:21:44 -0000 1.138 +++ nand_base.c 4 Apr 2005 18:02:23 -0000 1.139 @@ -1090,8 +1090,8 @@ * @len: number of bytes to read * @retlen: pointer to variable to store the number of read bytes * @buf: the databuffer to put data - * @oob_buf: filesystem supplied oob data buffer - * @oobsel: oob selection structure + * @oob_buf: filesystem supplied oob data buffer (can be NULL) + * @oobsel: oob selection structure (can be NULL) * @flags: flag to indicate if nand_get_device/nand_release_device should be preformed * and how many corrected error bits are acceptable: * bits 0..7 - number of tolerable errors @@ -1103,6 +1103,10 @@ size_t * retlen, u_char * buf, u_char * oob_buf, struct nand_oobinfo *oobsel, int flags) { + /* use userspace supplied oobinfo, if zero */ + if (oobsel == NULL) + oobsel = &mtd->oobinfo; + int i, j, col, realpage, page, end, ecc, chipnr, sndcmd = 1; int read = 0, oob = 0, ecc_status = 0, ecc_failed = 0; struct nand_chip *this = mtd->priv; @@ -1130,10 +1134,6 @@ if (flags & NAND_GET_DEVICE) nand_get_device (this, mtd, FL_READING); - /* use userspace supplied oobinfo, if zero */ - if (oobsel == NULL) - oobsel = &mtd->oobinfo; - /* Autoplace of oob data ? Use the default placement scheme */ if (oobsel->useecc == MTD_NANDECC_AUTOPLACE) oobsel = this->autooob; From gleixner at infradead.org Mon Apr 4 14:56:32 2005 From: gleixner at infradead.org (gleixner@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand nand_base.c,1.139,1.140 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv7955 Modified Files: nand_base.c Log Message: Move the NULL check into the calling function Index: nand_base.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/nand_base.c,v retrieving revision 1.139 retrieving revision 1.140 diff -u -r1.139 -r1.140 --- nand_base.c 4 Apr 2005 18:02:23 -0000 1.139 +++ nand_base.c 4 Apr 2005 18:56:29 -0000 1.140 @@ -1060,8 +1060,8 @@ */ static int nand_read (struct mtd_info *mtd, loff_t from, size_t len, size_t * retlen, u_char * buf) { - return nand_do_read_ecc (mtd, from, len, retlen, buf, NULL, NULL, 0xff); -} + return nand_do_read_ecc (mtd, from, len, retlen, buf, NULL, &mtd->oobinfo, 0xff); +} /** @@ -1079,6 +1079,9 @@ static int nand_read_ecc (struct mtd_info *mtd, loff_t from, size_t len, size_t * retlen, u_char * buf, u_char * oob_buf, struct nand_oobinfo *oobsel) { + /* use userspace supplied oobinfo, if zero */ + if (oobsel == NULL) + oobsel = &mtd->oobinfo; return nand_do_read_ecc(mtd, from, len, retlen, buf, oob_buf, oobsel, 0xff); } @@ -1091,7 +1094,7 @@ * @retlen: pointer to variable to store the number of read bytes * @buf: the databuffer to put data * @oob_buf: filesystem supplied oob data buffer (can be NULL) - * @oobsel: oob selection structure (can be NULL) + * @oobsel: oob selection structure * @flags: flag to indicate if nand_get_device/nand_release_device should be preformed * and how many corrected error bits are acceptable: * bits 0..7 - number of tolerable errors @@ -1103,10 +1106,7 @@ size_t * retlen, u_char * buf, u_char * oob_buf, struct nand_oobinfo *oobsel, int flags) { - /* use userspace supplied oobinfo, if zero */ - if (oobsel == NULL) - oobsel = &mtd->oobinfo; - + int i, j, col, realpage, page, end, ecc, chipnr, sndcmd = 1; int read = 0, oob = 0, ecc_status = 0, ecc_failed = 0; struct nand_chip *this = mtd->priv; From dedekind at infradead.org Tue Apr 5 08:51:58 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:45 2006 Subject: mtd/fs/jffs2 erase.c,1.74,1.75 wbuf.c,1.91,1.92 Message-ID: Update of /home/cvs/mtd/fs/jffs2 In directory phoenix.infradead.org:/tmp/cvs-serv11549 Modified Files: erase.c wbuf.c Log Message: Fix race problems with wbuf. The patch is accessible at http://lists.infradead.org/pipermail/linux-mtd/2005-April/012322.html Index: erase.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/erase.c,v retrieving revision 1.74 retrieving revision 1.75 diff -u -r1.74 -r1.75 --- erase.c 20 Mar 2005 17:46:20 -0000 1.74 +++ erase.c 5 Apr 2005 12:51:54 -0000 1.75 @@ -333,7 +333,11 @@ bad_offset = ofs; - ret = jffs2_flash_read(c, ofs, readlen, &retlen, ebuf); + if (!jffs2_is_writebuffered(c) || !jffs2_cleanmarker_oob(c)) + ret = c->mtd->read(c->mtd, ofs, readlen, &retlen, ebuf); + else + ret = c->mtd->read_ecc(c->mtd, ofs, readlen, &retlen, ebuf, NULL, c->oobinfo); + if (ret) { printk(KERN_WARNING "Read of newly-erased block at 0x%08x failed: %d. Putting on bad_list\n", ofs, ret); goto bad; Index: wbuf.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/wbuf.c,v retrieving revision 1.91 retrieving revision 1.92 diff -u -r1.91 -r1.92 --- wbuf.c 18 Mar 2005 09:58:06 -0000 1.91 +++ wbuf.c 5 Apr 2005 12:51:54 -0000 1.92 @@ -873,6 +873,7 @@ return c->mtd->read(c->mtd, ofs, len, retlen, buf); /* Read flash */ + down_read(&c->wbuf_sem); if (jffs2_cleanmarker_oob(c)) ret = c->mtd->read_ecc(c->mtd, ofs, len, retlen, buf, NULL, c->oobinfo); else @@ -896,16 +897,11 @@ /* if no writebuffer available or write buffer empty, return */ if (!c->wbuf_pagesize || !c->wbuf_len) - return ret;; + goto exit; /* if we read in a different block, return */ if (SECTOR_ADDR(ofs) != SECTOR_ADDR(c->wbuf_ofs)) - return ret; - - /* Lock only if we have reason to believe wbuf contains relevant data, - so that checking an erased block during wbuf recovery space allocation - does not deadlock. */ - down_read(&c->wbuf_sem); + goto exit; if (ofs >= c->wbuf_ofs) { owbf = (ofs - c->wbuf_ofs); /* offset in write buffer */ From dbrown at infradead.org Tue Apr 5 15:36:24 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/docboot doc_bootstub.S,1.5,1.6 doc_bootstub.h,1.5,1.6 Message-ID: Update of /home/cvs/mtd/docboot In directory phoenix.infradead.org:/tmp/cvs-serv12865 Modified Files: doc_bootstub.S doc_bootstub.h Log Message: Fix 16M limit test, add DEBUG_BUILD option. Index: doc_bootstub.S =================================================================== RCS file: /home/cvs/mtd/docboot/doc_bootstub.S,v retrieving revision 1.5 retrieving revision 1.6 diff -u -r1.5 -r1.6 --- doc_bootstub.S 29 Mar 2005 21:01:54 -0000 1.5 +++ doc_bootstub.S 5 Apr 2005 19:36:21 -0000 1.6 @@ -78,24 +78,9 @@ popw %ds MSG(kernel_string) -/* - mov $0x88, %ah - int $0x15 - call phword -*/ - cld - movw setup_seg, %ax - movw %ax, %es - - /* gdt_src is currently set to 0x8000. Add the setup base - address to it, yielding a 32k offset from the start of the - setup segment. The math is simplified by the fact that - setup_seg must be 1Kbyte aligned. */ - shrw $4, %ax - addw %ax, gdt_src_mid - + movw setup_seg, %es movw doc_seg, %ds movw $BXREG, %bx movw $SIREG, %si @@ -123,18 +108,22 @@ decw %cs:low_sects jnz read_setup_sects -#if 0 - /* Print the kernel version string. Partially for fun, mostly - to show that we've loaded the low part and are about to - load the high part. */ +#ifdef DEBUG_BUILD + /* Print the kernel version string. */ pushw %ds - pushw %es - popw %ds pushw %si pushw %bx + pushw %cs + popw %ds + MSG(crlf_string) + pushw %es + popw %ds movw kernel_version, %si addw $0x200, %si call message + pushw %cs + popw %ds + MSG(kernel_string) popw %bx popw %si popw %ds @@ -247,6 +236,18 @@ nextpage: popw %di doc_readpage: +#ifdef DEBUG_BUILD + pushw %bx + movw $4, %cx +bsloop: + movw $0x0001, %bx + movw $0x0e08, %ax + int $0x10 + loop bsloop + popw %bx + movw %dx, %ax + call phword +#endif #ifdef MILPLUS /* Flash command: Reset */ movb $NAND_CMD_RESET, %al @@ -375,7 +376,7 @@ ret /***************************************************************************/ -#if 0 +#ifdef DEBUG_BUILD phword: pushw %ax @@ -437,10 +438,18 @@ .word 0 .skip 0x10 +#ifdef DEBUG_BUILD +kernel_string: .string "\n\rLoading kernel... page 0x????" +initrd_string: .string " Loading initrd... page 0x????" +done_string: .string " done.\n\rCommandline: " +#else kernel_string: .string "Loading kernel... " initrd_string: .string "Loading initrd... " done_string: .string "done.\n\rCommandline: " +#endif + boot_string: .string "\n\rBooting!\n\r" +crlf_string = (. - 3) checksum: setup_seg: .word 0 @@ -499,13 +508,37 @@ MSG(installer_string) +#ifdef DEBUG_BUILD + MSG(docseg_string) + movw doc_seg, %ax + call phword + MSG(setupseg_string) +#endif + /* Store the setup segment */ - movw %cs, setup_seg + movw %cs, %ax + movw %ax, setup_seg +#ifdef DEBUG_BUILD + call phword +#endif + /* gdt_src is currently set to 0x8000. Add the setup base + address to it, yielding a 32k offset from the start of the + setup segment. The math is simplified by the fact that + setup_seg must be 1Kbyte aligned. */ + shrw $4, %ax + addw %ax, gdt_src_mid + +#ifdef DEBUG_BUILD + MSG(int1588_string) +#endif movb $0x88, %ah int $0x15 /* Get top of high mem (-1M) in 1k-blocks */ +#ifdef DEBUG_BUILD + call phword +#endif cmpw $(0x4000-0x400), %ax /* safety check: if it's above 15M ... */ - jae topmem_ok + jbe topmem_ok movw $(0x4000-0x400), %ax /* ... replace with 15M. */ topmem_ok: addw $0x400, %ax /* Adjust for 1M offset */ @@ -514,6 +547,13 @@ shlw $1, %ax /* Convert to 256-byte blocks */ andw $0xfff0, %ax /* Round down to 4k block (req'd by kernel) */ movw %ax, initrd_start /* Store initrd start for later use */ +#ifdef DEBUG_BUILD + pushw %ax + MSG(initrdstart_string) + popw %ax + call phword + MSG(lowmemtop_string) +#endif /* Now install the handler. What we need to do is: 1. Check the current end-of-memory in the BIOS @@ -525,6 +565,9 @@ /* Find top of low memory */ movw %dx, %ds movw 0x0413, %ax +#ifdef DEBUG_BUILD + call phword +#endif /* Steal enough room from the top. The line below is suboptimal for relocated code size of 2k bytes or less, but @@ -549,11 +592,38 @@ rep movsb +#ifdef DEBUG_BUILD + pushw %ax + MSG(handlerseg_string) + popw %ax + call phword + MSG(presskey_string) +keydrain: + movb $1, %ah + int $0x16 + movb $0, %ah + jz keywait + int $0x16 + jmp keydrain +keywait: + int $0x16 +#endif + skip_install: lret installer_string: .string "Installing DOCBoot.\n\r" +#ifdef DEBUG_BUILD +docseg_string: .string "doc_seg = 0x" +setupseg_string: .string " setup_seg = 0x" +int1588_string: .string "\n\rint15/88 returns 0x" +initrdstart_string: .string " initrd_start = 0x" +lowmemtop_string: .string "00\n\rtop of low mem = 0x" +handlerseg_string: .string "k handler seg = 0x" +presskey_string: .string "\n\r -- Press any key --\n\r" +#endif + install_end: .balign 512, 0xff /* Pad the entire stub to a 512-byte boundary */ Index: doc_bootstub.h =================================================================== RCS file: /home/cvs/mtd/docboot/doc_bootstub.h,v retrieving revision 1.5 retrieving revision 1.6 diff -u -r1.5 -r1.6 --- doc_bootstub.h 29 Mar 2005 21:07:28 -0000 1.5 +++ doc_bootstub.h 5 Apr 2005 19:36:21 -0000 1.6 @@ -155,6 +155,7 @@ /* #define BIOS_EXTENSION */ /* #define MILPLUS */ /* #define OLD_DOC2K */ +/* #define DEBUG_BUILD */ #ifdef OLD_DOC2K #define CDSN_CTRL_BASE CDSN_CTRL_FLASH_IO + CDSN_CTRL_WP + CDSN_CTRL_CE From dedekind at infradead.org Wed Apr 6 12:02:55 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:45 2006 Subject: mtd/fs/jffs2 nodemgmt.c,1.120,1.121 Message-ID: Update of /home/cvs/mtd/fs/jffs2 In directory phoenix.infradead.org:/tmp/cvs-serv16890 Modified Files: nodemgmt.c Log Message: Add KERN_DEBUG level to printks Index: nodemgmt.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/nodemgmt.c,v retrieving revision 1.120 retrieving revision 1.121 diff -u -r1.120 -r1.121 --- nodemgmt.c 20 Mar 2005 17:46:20 -0000 1.120 +++ nodemgmt.c 6 Apr 2005 16:02:52 -0000 1.121 @@ -437,7 +437,7 @@ // Take care, that wasted size is taken into concern if ((jeb->dirty_size || ISDIRTY(jeb->wasted_size + ref_totlen(c, jeb, ref))) && jeb != c->nextblock) { - D1(printk("Dirtying\n")); + D1(printk(KERN_DEBUG "Dirtying\n")); addedsize = ref_totlen(c, jeb, ref); jeb->dirty_size += ref_totlen(c, jeb, ref); c->dirty_size += ref_totlen(c, jeb, ref); @@ -459,7 +459,7 @@ } } } else { - D1(printk("Wasting\n")); + D1(printk(KERN_DEBUG "Wasting\n")); addedsize = 0; jeb->wasted_size += ref_totlen(c, jeb, ref); c->wasted_size += ref_totlen(c, jeb, ref); From dbrown at infradead.org Wed Apr 6 14:10:24 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand diskonchip.c,1.50,1.51 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv17428 Modified Files: diskonchip.c Log Message: Fix (?) free OOB array info. I really hope this doesn't break something. Index: diskonchip.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/diskonchip.c,v retrieving revision 1.50 retrieving revision 1.51 diff -u -r1.50 -r1.51 --- diskonchip.c 29 Mar 2005 20:57:45 -0000 1.50 +++ diskonchip.c 6 Apr 2005 18:10:20 -0000 1.51 @@ -1055,7 +1055,7 @@ .useecc = MTD_NANDECC_AUTOPLACE, .eccbytes = 6, .eccpos = {0, 1, 2, 3, 4, 5}, - .oobfree = { {8, 8} } + .oobfree = { {6, 10} } }; /* Find the (I)NFTL Media Header, and optionally also the mirror media header. From dbrown at infradead.org Wed Apr 6 16:13:09 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand nand_base.c,1.140,1.141 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv17791 Modified Files: nand_base.c Log Message: Fix reading of autoplaced OOB when there are multiple free sections. Index: nand_base.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/nand_base.c,v retrieving revision 1.140 retrieving revision 1.141 diff -u -r1.140 -r1.141 --- nand_base.c 4 Apr 2005 18:56:29 -0000 1.140 +++ nand_base.c 6 Apr 2005 20:13:05 -0000 1.141 @@ -1285,13 +1285,12 @@ switch(oobsel->useecc) { case MTD_NANDECC_AUTOPLACE: /* Walk through the autoplace chunks */ - for (i = 0, j = 0; j < mtd->oobavail; i++) { + for (i = 0; oobsel->oobfree[i][1]; i++) { int from = oobsel->oobfree[i][0]; int num = oobsel->oobfree[i][1]; memcpy(&oob_buf[oob], &oob_data[from], num); - j+= num; + oob += num; } - oob += mtd->oobavail; break; case MTD_NANDECC_PLACE: /* YAFFS1 legacy mode */ From dbrown at infradead.org Wed Apr 6 16:14:22 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand diskonchip.c,1.51,1.52 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv17809 Modified Files: diskonchip.c Log Message: Try not to break existing jffs2 installs, instead break oobfree into two out-of-order pieces. Index: diskonchip.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/diskonchip.c,v retrieving revision 1.51 retrieving revision 1.52 diff -u -r1.51 -r1.52 --- diskonchip.c 6 Apr 2005 18:10:20 -0000 1.51 +++ diskonchip.c 6 Apr 2005 20:14:19 -0000 1.52 @@ -1055,7 +1055,7 @@ .useecc = MTD_NANDECC_AUTOPLACE, .eccbytes = 6, .eccpos = {0, 1, 2, 3, 4, 5}, - .oobfree = { {6, 10} } + .oobfree = { {8, 8}, {6, 2} } }; /* Find the (I)NFTL Media Header, and optionally also the mirror media header. From dbrown at infradead.org Thu Apr 7 09:39:17 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand diskonchip.c,1.52,1.53 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv21245 Modified Files: diskonchip.c Log Message: Fix compile w/o CONFIG_MTD_PARTITIONS. Index: diskonchip.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/diskonchip.c,v retrieving revision 1.52 retrieving revision 1.53 diff -u -r1.52 -r1.53 --- diskonchip.c 6 Apr 2005 20:14:19 -0000 1.52 +++ diskonchip.c 7 Apr 2005 13:39:13 -0000 1.53 @@ -106,13 +106,11 @@ static int no_ecc_failures=0; module_param(no_ecc_failures, int, 0); -#ifdef CONFIG_MTD_PARTITIONS static int no_autopart=0; module_param(no_autopart, int, 0); static int show_firmware_partition=0; module_param(show_firmware_partition, int, 0); -#endif #ifdef MTD_NAND_DISKONCHIP_BBTWRITE static int inftl_bbt_write=1; From dbrown at infradead.org Thu Apr 7 10:17:49 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/util nandwrite.c,1.29,1.30 Message-ID: Update of /home/cvs/mtd/util In directory phoenix.infradead.org:/tmp/cvs-serv21394 Modified Files: nandwrite.c Log Message: Support out-of-order oobfree segments; used by diskonchip. Index: nandwrite.c =================================================================== RCS file: /home/cvs/mtd/util/nandwrite.c,v retrieving revision 1.29 retrieving revision 1.30 diff -u -r1.29 -r1.30 --- nandwrite.c 24 Mar 2005 20:42:05 -0000 1.29 +++ nandwrite.c 7 Apr 2005 14:17:46 -0000 1.30 @@ -48,6 +48,7 @@ */ unsigned char writebuf[MAX_PAGE_SIZE]; unsigned char oobbuf[MAX_OOB_SIZE]; +unsigned char oobreadbuf[MAX_OOB_SIZE]; // oob layouts to pass into the kernel as default struct nand_oobinfo none_oobinfo = { @@ -215,6 +216,8 @@ process_options(argc, argv); + memset(oobbuf, 0xff, sizeof(oobbuf)); + if (pad && writeoob) { fprintf(stderr, "Can't pad when oob data is present.\n"); exit(1); @@ -304,7 +307,7 @@ } oob.length = meminfo.oobsize; - oob.ptr = oobbuf; + oob.ptr = noecc ? oobreadbuf : oobbuf; /* Open the input file */ if ((ifd = open(img, O_RDONLY)) == -1) { @@ -384,29 +387,35 @@ if (writeoob) { /* Read OOB data from input file, exit on failure */ - if ((cnt = read(ifd, oobbuf, meminfo.oobsize)) != meminfo.oobsize) { + if ((cnt = read(ifd, oobreadbuf, meminfo.oobsize)) != meminfo.oobsize) { perror ("File I/O error on input file"); goto closeall; } if (!noecc) { - int i, j; + int i, start, len; /* * We use autoplacement and have the oobinfo with the autoplacement * information from the kernel available + * + * Modified to support out of order oobfree segments, + * such as the layout used by diskonchip.c */ if (!oobinfochanged && (old_oobinfo.useecc == MTD_NANDECC_AUTOPLACE)) { - j = 0; for (i = 0;old_oobinfo.oobfree[i][1]; i++) { /* Set the reserved bytes to 0xff */ - for (; j < old_oobinfo.oobfree[i][0]; j++) - oobbuf[j] = 0xff; - /* Set j to the end of the free area */ - j = old_oobinfo.oobfree[i][0] + old_oobinfo.oobfree[i][1]; + start = old_oobinfo.oobfree[i][0]; + len = old_oobinfo.oobfree[i][1]; + memcpy(oobbuf + start, + oobreadbuf + start, + len); } } else { /* Set at least the ecc byte positions to 0xff */ - for (i = 0; i < old_oobinfo.eccbytes; i++) - oobbuf[old_oobinfo.eccpos[i]] = 0xff; + start = old_oobinfo.eccbytes; + len = meminfo.oobsize - start; + memcpy(oobbuf + start, + oobreadbuf + start, + len); } } /* Write OOB data first, as ecc will be placed in there*/ From dbrown at infradead.org Thu Apr 7 10:22:58 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:45 2006 Subject: mtd/drivers/mtd/nand diskonchip.c,1.53,1.54 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv21422 Modified Files: diskonchip.c Log Message: Add helpful comment about oobfree so I can't claim two years from now that I don't remember what I was thinking. Index: diskonchip.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/diskonchip.c,v retrieving revision 1.53 retrieving revision 1.54 diff -u -r1.53 -r1.54 --- diskonchip.c 7 Apr 2005 13:39:13 -0000 1.53 +++ diskonchip.c 7 Apr 2005 14:22:55 -0000 1.54 @@ -1049,6 +1049,16 @@ //u_char mydatabuf[528]; +/* The strange out-of-order .oobfree list below is a (possibly unneeded) + * attempt to retain compatibility. It used to read: + * .oobfree = { {8, 8} } + * Since that leaves two bytes unusable, it was changed. But the following + * scheme might affect existing jffs2 installs by moving the cleanmarker: + * .oobfree = { {6, 10} } + * jffs2 seems to handle the above gracefully, but the current scheme seems + * safer. The only problem with it is that any code that parses oobfree must + * be able to handle out-of-order segments. + */ static struct nand_oobinfo doc200x_oobinfo = { .useecc = MTD_NANDECC_AUTOPLACE, .eccbytes = 6, From dedekind at infradead.org Sat Apr 9 06:47:03 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:46 2006 Subject: mtd/fs/jffs2 nodelist.h,1.129,1.130 gc.c,1.147,1.148 Message-ID: Update of /home/cvs/mtd/fs/jffs2 In directory phoenix.infradead.org:/tmp/cvs-serv30067 Modified Files: nodelist.h gc.c Log Message: Fix the race problem described here: http://lists.infradead.org/pipermail/linux-mtd/2005-April/012361.html Index: nodelist.h =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/nodelist.h,v retrieving revision 1.129 retrieving revision 1.130 diff -u -r1.129 -r1.130 --- nodelist.h 17 Mar 2005 20:23:06 -0000 1.129 +++ nodelist.h 9 Apr 2005 10:46:59 -0000 1.130 @@ -363,6 +363,18 @@ node = node->rb_left; return rb_entry(node, struct jffs2_node_frag, rb); } + +static inline struct jffs2_node_frag *frag_last(struct rb_root *root) +{ + struct rb_node *node = root->rb_node; + + if (!node) + return NULL; + while(node->rb_right) + node = node->rb_right; + return rb_entry(node, struct jffs2_node_frag, rb); +} + #define rb_parent(rb) ((rb)->rb_parent) #define frag_next(frag) rb_entry(rb_next(&(frag)->rb), struct jffs2_node_frag, rb) #define frag_prev(frag) rb_entry(rb_prev(&(frag)->rb), struct jffs2_node_frag, rb) Index: gc.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/gc.c,v retrieving revision 1.147 retrieving revision 1.148 diff -u -r1.147 -r1.148 --- gc.c 20 Mar 2005 21:43:22 -0000 1.147 +++ gc.c 9 Apr 2005 10:47:00 -0000 1.148 @@ -669,9 +669,10 @@ { struct jffs2_full_dnode *new_fn; struct jffs2_raw_inode ri; + struct jffs2_node_frag *last_frag; jint16_t dev; char *mdata = NULL, mdatalen = 0; - uint32_t alloclen, phys_ofs; + uint32_t alloclen, phys_ofs, ilen; int ret; if (S_ISBLK(JFFS2_F_I_MODE(f)) || @@ -707,6 +708,14 @@ goto out; } + last_frag = frag_last(&f->fragtree); + if (last_frag) + /* Fetch the inode length from the fragtree rather then + * from i_size since i_size may have not been updated yet */ + ilen = last_frag->ofs + last_frag->size; + else + ilen = JFFS2_F_I_SIZE(f); + memset(&ri, 0, sizeof(ri)); ri.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); ri.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE); @@ -718,7 +727,7 @@ ri.mode = cpu_to_jemode(JFFS2_F_I_MODE(f)); ri.uid = cpu_to_je16(JFFS2_F_I_UID(f)); ri.gid = cpu_to_je16(JFFS2_F_I_GID(f)); - ri.isize = cpu_to_je32(JFFS2_F_I_SIZE(f)); + ri.isize = cpu_to_je32(ilen); ri.atime = cpu_to_je32(JFFS2_F_I_ATIME(f)); ri.ctime = cpu_to_je32(JFFS2_F_I_CTIME(f)); ri.mtime = cpu_to_je32(JFFS2_F_I_MTIME(f)); @@ -898,7 +907,7 @@ struct jffs2_raw_inode ri; struct jffs2_node_frag *frag; struct jffs2_full_dnode *new_fn; - uint32_t alloclen, phys_ofs; + uint32_t alloclen, phys_ofs, ilen; int ret; D1(printk(KERN_DEBUG "Writing replacement hole node for ino #%u from offset 0x%x to 0x%x\n", @@ -958,10 +967,19 @@ ri.csize = cpu_to_je32(0); ri.compr = JFFS2_COMPR_ZERO; } + + frag = frag_last(&f->fragtree); + if (frag) + /* Fetch the inode length from the fragtree rather then + * from i_size since i_size may have not been updated yet */ + ilen = frag->ofs + frag->size; + else + ilen = JFFS2_F_I_SIZE(f); + ri.mode = cpu_to_jemode(JFFS2_F_I_MODE(f)); ri.uid = cpu_to_je16(JFFS2_F_I_UID(f)); ri.gid = cpu_to_je16(JFFS2_F_I_GID(f)); - ri.isize = cpu_to_je32(JFFS2_F_I_SIZE(f)); + ri.isize = cpu_to_je32(ilen); ri.atime = cpu_to_je32(JFFS2_F_I_ATIME(f)); ri.ctime = cpu_to_je32(JFFS2_F_I_CTIME(f)); ri.mtime = cpu_to_je32(JFFS2_F_I_MTIME(f)); @@ -1168,7 +1186,7 @@ D1(printk(KERN_DEBUG "Expanded dnode to write from (0x%x-0x%x) to (0x%x-0x%x)\n", orig_start, orig_end, start, end)); - BUG_ON(end > JFFS2_F_I_SIZE(f)); + D1(BUG_ON(end > frag_last(&f->fragtree)->ofs + frag_last(&f->fragtree)->size)); BUG_ON(end < orig_end); BUG_ON(start > orig_start); } From gleixner at infradead.org Mon Apr 11 06:19:05 2005 From: gleixner at infradead.org (gleixner@infradead.org) Date: Mon May 15 21:51:46 2006 Subject: mtd/include/linux/mtd mtd.h,1.58,1.59 Message-ID: Update of /home/cvs/mtd/include/linux/mtd In directory phoenix.infradead.org:/tmp/cvs-serv22009 Modified Files: mtd.h Log Message: Move kernel data where it belongs. Previous change broke user abi. Index: mtd.h =================================================================== RCS file: /home/cvs/mtd/include/linux/mtd/mtd.h,v retrieving revision 1.58 retrieving revision 1.59 diff -u -r1.58 -r1.59 --- mtd.h 1 Apr 2005 01:59:54 -0000 1.58 +++ mtd.h 11 Apr 2005 10:19:02 -0000 1.59 @@ -70,7 +70,6 @@ u_int32_t oobblock; // Size of OOB blocks (e.g. 512) u_int32_t oobsize; // Amount of OOB data per block (e.g. 16) - u_int32_t oobavail; // Number of bytes in OOB area available for fs u_int32_t ecctype; u_int32_t eccsize; @@ -81,6 +80,7 @@ // oobinfo is a nand_oobinfo structure, which can be set by iotcl (MEMSETOOBINFO) struct nand_oobinfo oobinfo; + u_int32_t oobavail; // Number of bytes in OOB area available for fs /* Data for variable erase regions. If numeraseregions is zero, * it means that the whole device has erasesize as given above. From lavinen at infradead.org Mon Apr 11 10:16:11 2005 From: lavinen at infradead.org (lavinen@infradead.org) Date: Mon May 15 21:51:46 2006 Subject: mtd/drivers/mtd/nand nand_base.c,1.141,1.142 Message-ID: Update of /home/cvs/mtd/drivers/mtd/nand In directory phoenix.infradead.org:/tmp/cvs-serv23051 Modified Files: nand_base.c Log Message: Reverting the change from 1.136 to 1.137 (back to static allocation of ecc arrays) due to stack corruption and ecc errors. Index: nand_base.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/nand/nand_base.c,v retrieving revision 1.141 retrieving revision 1.142 diff -u -r1.141 -r1.142 --- nand_base.c 6 Apr 2005 20:13:05 -0000 1.141 +++ nand_base.c 11 Apr 2005 14:16:07 -0000 1.142 @@ -855,7 +855,7 @@ u_char *oob_buf, struct nand_oobinfo *oobsel, int cached) { int i, status; - u_char ecc_code[oobsel->eccbytes]; + u_char ecc_code[32]; int eccmode = oobsel->useecc ? this->eccmode : NAND_ECC_NONE; int *oob_config = oobsel->eccpos; int datidx = 0, eccidx = 0, eccsteps = this->eccsteps; @@ -961,7 +961,7 @@ int i, j, datidx = 0, oobofs = 0, res = -EIO; int eccsteps = this->eccsteps; int hweccbytes; - u_char oobdata[mtd->oobsize]; + u_char oobdata[64]; hweccbytes = (this->options & NAND_HWECC_SYNDROME) ? (oobsel->eccbytes / eccsteps) : 0; @@ -1111,8 +1111,8 @@ int read = 0, oob = 0, ecc_status = 0, ecc_failed = 0; struct nand_chip *this = mtd->priv; u_char *data_poi, *oob_data = oob_buf; - u_char ecc_calc[oobsel->eccbytes]; - u_char ecc_code[oobsel->eccbytes]; + u_char ecc_calc[32]; + u_char ecc_code[32]; int eccmode, eccsteps; int *oob_config, datidx; int blockcheck = (1 << (this->phys_erase_shift - this->page_shift)) - 1; From havasi at infradead.org Tue Apr 12 08:48:59 2005 From: havasi at infradead.org (havasi@infradead.org) Date: Mon May 15 21:51:46 2006 Subject: mtd/html/images havasi.jpg,NONE,1.1 Message-ID: Update of /home/cvs/mtd/html/images In directory phoenix.infradead.org:/tmp/cvs-serv27288 Added Files: havasi.jpg Log Message: Photo of Ferenc Havasi --- NEW FILE havasi.jpg --- JFIFHHExifII* z(2iCanonCanon PowerShot S1 IS2004:09:27 13:56:324<0220DXl t|  Y |0100< 2004:09:27 13:56:322004:09:27 13:56:32 _ _ .b" *BJc  y6 H\ _  Dx_a xrNIMG:PowerShot S1 IS JPEGFirmware Version 1.10i  ji%jrv~pj{@%h@pR980100]e(m4JFIFC    $.' ",#(7),01444'9=82<.342C  2!!22222222222222222222222222222222222222222222222222" }!1AQa"q2#BR$3br %&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz w!1AQaq"2B #3Rbr $4%&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz ?JpVί_ʗ_~U)1zv9_|4sUl')c+HsZgAԇ]>iu\߳JgϣަI5j.qއ$2n/g"\ S,*CelkмMr?*qոž+aHY$߼v*\%sz)*m@&$SNjwHdϻԹ@Hި-M[c[*P?0@!\(q4帕w(JpJ`G*gXehiݩi B;?OZhV̯&n~DδE4.k{kQXylxG˼V<(f$SNUl,H[ʄi* qPj7"08L?=*PEdMZ%ןnCхzCچ3ȯ'1Sw< AFxajGFTImchZFO = l$~CɑܡU5i%FNjNg#H)__Fhۜʫn 3)4J?tgRqUc挮 |j̧v#͜3! *3%vS[1|U~I'i34Gcg2b0Il WG5FfTqm,urkb;ZVf*'%|ik}: [P3MTsX&G"yt`w7k[SNxOPW,SjF+Ũ63^ HPtEj̙UTNiSSVP*APsR`CERSdu6v?*ug} P@+ ~C?['T|?ndY=I1>ڗ! EP\gU#T8vus.R{RiI\m+(jVcq˕ig#Q\k!'itcGknʼn*zk\AׁS5Pgr9 IWKoiBXă[ǿXkwlɞIY.24]I5g5U.`j;KE]3SzBPb-Vj p?t^ز͌­g@y&{xtZS\5LWJdNq:TMJ=i((ıy^!b~">oY`$I]Svg1ɇ2XG'5ydQbv5gr=}j) w#]-٢wn!`7Ss[W5|!q'q,G,F18.fֹrNM/j2sUC,zbIrH>^D𰕉6}1 "mwN7`^8w&K+:鹎#k ~dsDr!tD_D51֐ ăw;R L-`'I;FQmM48^W<6fpOא:6| Ff1LzsRS4m9c`IIF oKc᷁rrڭh Y4PȜ[#XqV>q2OGNi=[#i7>?Fg M\P wCugt7ƿ4c8YWڷړC];$]3É\y=͗6Z<2fsI=~bn$'"tܕG "*宨246R5cDpVZ+_ I#VvE?p;]b@DiZfL'F4n<)8.."q!iԑR/uskno=6*)$'Śn yQeG+3H">R0]6{_֫ޥM&MIBv Vږ 9X뷺-2%'ŗ}I+\{f țDn?DZ;#DJ&Vyph? KlF^pQN;fSXC@sdkZ wm7 I-cp7I5y1sn(&pֻq|unC_T7c鎂KhDnvT-4dU}MȍP$ DguLHrKeDH  qMo9M`5;tqgyuv.AIǙ66GS-&!VOzF$p2'YGjMufC& ۹;&Y1薖RBJDmICY3l#7mXV.:\l-^>3k.ÌǮ9hCu04I"8HnMoa֚NrH ߣg 7.+#9'z eCߒ[bI`$X<3u $JARg#skKjBwB|w׺2txs3㈛0IT~FD ˋ˘sP{}l=l-Hs4,U_N#? ŏ2fÞ뭽 7Hi,ۃM{:wbCt.$5L nA`4,wXaHwի^[c,7l@.$z"x;$srs4V=:?`hO~(xU m})Ҹ6Hu՟J(UIч bn>\-í BVg+QH^ߺQ% MO 찛2f]6bm}P&"K~ȿOY+X\w>yЅ!l8UxۧgcWȭUqN[eèY1s@pv躉մFH= Zfks@'[xmg18΂6v(n!HI-\kcVXDx&,ܠVܯbc\lB (R!OL-&n6L6:p6t&6Z(`4c`Bs;tlR a4m}U^HN*e%oFz fU遤zR2TOq臌U, 'p1CTiCZ+KEaw#fO)=דܧߠ D!:Jj|eܬ7nߖSvU> .v;("LQҞd`p>_HgPT3.6,O_6>Xn.w٪) s~o'ִ_bAI0BQzI4ǒn$CZ2KúG0{lT{t)> \#ď&I[] g-ˋ8g lw{D^߸]n`4PqYclwi$ .#@kK/]KKR+M ;fLO"gXC,CXOArZ1vhcI`אkH F)$п$Ͽ\KŐ<_.K긯Yk~#tY42si-|R7t8x#ZXH$uC٦dK;+ahuޭ ܅EXU>HzTv0v< u-z#2h!n[\LhoEc1>V;bdPުŮ\%vW {Ql(S+v#Zasm[;PlMpf 4HʔʛIqkIڊڗG017 -m?&3 QbɺeYui5wR/ sp˵ﲈ$8 Y8CƗ̶&|>UOcُ,99,g+%]`~>(x%\ .wdb&Ƞ=VdܴjcULj/s;ޣA$̊Vm>Wvttk&\lMU.S:uae|@1d䎥5. hiڃ98P;ZFV7IDfQ7O/ZvBZ#i:5(\<~_b5H;&⁵ٯ*6I$$I!GJ9!sA?fW]_kڗq G:hHh<â,9a/PDǃ܃G*Zm&-q# v$}OLv9Z,yJ,m4FNSƕ(̍)a4l 6㌨BPl`|UXTٹ|4nyh9$iI\͟38}6U$nb\֝+x,$V5l[ooD7Y6QhqTe W-?yR#j4qnm{.FPs@(&[! PH@k^;ƿU7śS-OkXi$j-$B0I$.T> EX,l4 ԶoPZmW֊^n.yb8qBg3jN  hMV4F@K"eYE]<H V 3q^KN5t>cTW:Fߚ[D8[*HsURb,cP&B.WXɞվ|`o;fi%rzV>]Cﯩ WIHPVaz^/vRF% sk;^0e6|݁IQG|iyghQa$4GR[xk ty7ыO ⇈ Bfە;k EdZOIk[@6e5h<0t<3{+H%8֒z}IIeox'S$J^^HBI$x'Ƒ͗/]pDǼDޙR'Rex* ,t/A+ewR9hOhI)(>K0!$I$M͑;9䑬h2s\ˏ@wk5.i^F :X 88\}I= (`1CHܨK舲;YVb{F8-g0qץ=2~;5|J+`;b<γDJe~,嵎#U,@?͑ܤ9 so}b*#,RFG#{w+UC&Z]Ĝ8"sC}V^>.<,{L}xZj&<%/ە!boE3b#ؐa} QD`6&V_c&u%XhMj%F&m +t/۲Qt { M?Q17 Dyn{~o f2c:yq<lw ɥz\e9k{l<6u% :k@<]( Update of /home/cvs/mtd/html In directory phoenix.infradead.org:/tmp/cvs-serv27472 Modified Files: fellows.xml Log Message: Add myself (Ferenc Havasi). Index: fellows.xml =================================================================== RCS file: /home/cvs/mtd/html/fellows.xml,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- fellows.xml 12 Mar 2005 13:48:21 -0000 1.1 +++ fellows.xml 12 Apr 2005 12:58:56 -0000 1.2 @@ -40,6 +40,13 @@

Artem B. Bityuckiy

Artem is involved in JFFS2 and started the development of JFFS3 recently

+
Image of Ferenc Havasi +

Ferenc Havasi

+

Ferenc has rewritten the compression code of JFFS2 (not fully finished yet) and made mount time + improvement called summary. It is included in JFFS3 and there is an unofficial patch for JFFS2, too. + He works at the University of Szeged in Hungary, and there is a team behind him: Zoltan Sogor and Patrik Kluba. +

+

More to come

Unknown, forgotten ...

From dedekind at infradead.org Tue Apr 12 12:45:54 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:46 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.32,1.33 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv28130 Modified Files: JFFS3design.tex Log Message: Add a chapter with digits. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.32 retrieving revision 1.33 diff -u -r1.32 -r1.33 --- JFFS3design.tex 30 Mar 2005 13:27:18 -0000 1.32 +++ JFFS3design.tex 12 Apr 2005 16:45:51 -0000 1.33 @@ -30,6 +30,9 @@ \begin{abstract} This document discusses various JFFS3 high-level design aspects. Additionally, it defines standard JFFS3 dictionary and terms. + +The document assumes the reader is familiar with JFFS2 design and has at +least read David Woodhouse's JFFS2 paper [\ref{ref_JFFSdwmw2}]. \end{abstract} % @@ -46,6 +49,100 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % +% JFFS2 problems analysis +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\section{JFFS2 analysis} +This section analyzes JFFS2 disadvantages in order to motivate JFFS3 +improvements. + +% +% Memory consumption +% +\subsection{Memory consumption} + +% +% Real-live experiment +% +\subsubsection{Real-life experiment} +The following experiment was done: +\begin{itemize} +\item an x86 PC machine with 64 MiB of simulated NAND flash (nandsim module) was +used; + +\item a typical Linux root file system was put on the flash, namely +\texttt{/bin}, \texttt{/sbin}, \texttt{/etc}, \texttt{/boot} and +partially \texttt{/usr} from the x86 Fedora Core 2 distribution; + +\item the total files number was 4372 - 719 directories and 2995 regular files; + +\item all files' nodes were pristine; + +\item the summary size of all files was 116 MiB (compression was +enabled). +\end{itemize} + +The memory consumed by JFFS2 was distributed over its internal objects +as follows. + +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 658 KiB \\ +jffs2\_node\_frag & 913 KiB \\ +jffs2\_full\_dnode & 545 KiB \\ +jffs2\_full\_dirent & 102 KiB \\ +\hline +total & 2.2 MiB +\end{tabular} +\end{center} + +If we run the following command on the same empty 64 MiB JFFS2 file +system +\begin{quote} +\texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=512} +\end{quote} +we'll get the following result: + +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 1778 KiB \\ +jffs2\_node\_frag & 3111 KiB \\ +jffs2\_full\_dnode & 1777 KiB \\ +\hline +total & 6.7 MiB +\end{tabular} +\end{center} + +And the command +\begin{quote} +\texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=10} +\end{quote} +gives: + +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 12706 KiB \\ +jffs2\_node\_frag & 22200 KiB \\ +jffs2\_full\_dnode & 12686 KiB \\ +\hline +total & 47.6 MiB +\end{tabular} +\end{center} + +Note, that all inodes were in the inode cache in our experiment, +which isn't that typical +for the real-life system though. If no inodes were in the inode cache, +only 658/1778/12706 KiB would be consumed by \texttt{jffs2\_node\_ref} +objects. However, opening any +file or looking up in any directory would require additional RAM. + +It is worth noting here, that in JFFS2 memory which is consumed even in +the case when no files are opened is called \emph{in-core memory}. +In-core memory mostly consists of \texttt{jffs2\_node\_ref} objects. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% % CHECKSUM % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% From dedekind at infradead.org Wed Apr 13 09:14:34 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.33,1.34 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv32039 Modified Files: JFFS3design.tex Log Message: Reformat. Don't use upsetting "JFFS2 problems" phrase. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.33 retrieving revision 1.34 diff -u -r1.33 -r1.34 --- JFFS3design.tex 12 Apr 2005 16:45:51 -0000 1.33 +++ JFFS3design.tex 13 Apr 2005 13:14:31 -0000 1.34 @@ -49,7 +49,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % -% JFFS2 problems analysis +% JFFS2 analysis % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{JFFS2 analysis} @@ -60,27 +60,20 @@ % Memory consumption % \subsection{Memory consumption} +Let's make experiments using an x86 PC host with 64MiB simulated +NAND flash (nandsim module). In all experiments below the whole +flash comprises single partition where JFFS2 filesystem is put. -% -% Real-live experiment -% -\subsubsection{Real-life experiment} -The following experiment was done: -\begin{itemize} -\item an x86 PC machine with 64 MiB of simulated NAND flash (nandsim module) was -used; +\begin{flushleft}\textbf{Experiment 1}\end{flushleft} -\item a typical Linux root file system was put on the flash, namely +Fulfill the JFFS2 file system with a typical Linux root FS, namely \texttt{/bin}, \texttt{/sbin}, \texttt{/etc}, \texttt{/boot} and -partially \texttt{/usr} from the x86 Fedora Core 2 distribution; - -\item the total files number was 4372 - 719 directories and 2995 regular files; +partially \texttt{/usr} from the x86 Fedora Core 2 distribution. -\item all files' nodes were pristine; - -\item the summary size of all files was 116 MiB (compression was -enabled). -\end{itemize} +We observed the following. +The total files number was 4372 - 719 directories and 2995 regular +files (all files' nodes were made pristine). The summary size of all files was 116 MiB. +The summary size of all files was 116 MiB (compression was enabled). The memory consumed by JFFS2 was distributed over its internal objects as follows. @@ -96,12 +89,13 @@ \end{tabular} \end{center} -If we run the following command on the same empty 64 MiB JFFS2 file +\begin{flushleft}\textbf{Experiment 2}\end{flushleft} +The following command on the same empty 64 MiB JFFS2 file system \begin{quote} \texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=512} \end{quote} -we'll get the following result: +gave the following results: \begin{center} \begin{tabular}{ll} @@ -113,11 +107,13 @@ \end{tabular} \end{center} -And the command +\begin{flushleft}\textbf{Experiment 3}\end{flushleft} +The following command on the same empty 64 MiB JFFS2 file +system \begin{quote} \texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=10} \end{quote} -gives: +gave: \begin{center} \begin{tabular}{ll} @@ -129,10 +125,10 @@ \end{tabular} \end{center} -Note, that all inodes were in the inode cache in our experiment, +Note, that all inodes were in the inode cache in our experiments, which isn't that typical for the real-life system though. If no inodes were in the inode cache, -only 658/1778/12706 KiB would be consumed by \texttt{jffs2\_node\_ref} +only 658/1778/12706 KiB would be consumed by the \texttt{jffs2\_node\_ref} objects. However, opening any file or looking up in any directory would require additional RAM. @@ -140,6 +136,45 @@ the case when no files are opened is called \emph{in-core memory}. In-core memory mostly consists of \texttt{jffs2\_node\_ref} objects. +Assuming the amount consumed memory grows linearly with the flash size +(which I believe is true) we'd get the following numbers for 1GB flash +in the same experiments: + +\begin{flushleft}\textbf{Experiment 1a (imaginary)}\end{flushleft} +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 10.3 MiB \\ +jffs2\_node\_frag & 14.3 MiB \\ +jffs2\_full\_dnode & 8.5 MiB \\ +jffs2\_full\_dirent & 1.6 MiB \\ +\hline +total & 34.7 MiB +\end{tabular} +\end{center} + +\begin{flushleft}\textbf{Experiment 2a (imaginary)}\end{flushleft} +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 27.8 KiB \\ +jffs2\_node\_frag & 48.6 KiB \\ +jffs2\_full\_dnode & 27.8 KiB \\ +\hline +total & 104.2 MiB +\end{tabular} +\end{center} + +\begin{flushleft}\textbf{Experiment 3a (imaginary)}\end{flushleft} +\begin{center} +\begin{tabular}{ll} +jffs2\_node\_ref & 198.5 KiB \\ +jffs2\_node\_frag & 346.9 KiB \\ +jffs2\_full\_dnode & 198.2 KiB \\ +\hline +total & 743.6 MiB +\end{tabular} +\end{center} + +Needless to note that this is unacceptable for embedded systems. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dwmw2 at infradead.org Wed Apr 13 09:22:38 2005 From: dwmw2 at infradead.org (David Woodhouse) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs2 README.Locking, 1.11, 1.12 nodelist.c, 1.93, 1.94 write.c, 1.91, 1.92 Message-ID: Update of /home/cvs/mtd/fs/jffs2 In directory phoenix.infradead.org:/tmp/cvs-serv32133 Modified Files: README.Locking nodelist.c write.c Log Message: Fix inode allocation race Index: README.Locking =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/README.Locking,v retrieving revision 1.11 retrieving revision 1.12 diff -u -r1.11 -r1.12 --- README.Locking 24 Nov 2004 09:25:09 -0000 1.11 +++ README.Locking 13 Apr 2005 13:22:35 -0000 1.12 @@ -108,6 +108,10 @@ correspondent jffs2_inode_cache object). So, the inocache_lock has to be locked while walking the c->inocache_list hash buckets. +This spinlock also covers allocation of new inode numbers, which is +currently just '++->highest_ino++', but might one day get more complicated +if we need to deal with wrapping after 4 milliard inode numbers are used. + Note, the f->sem guarantees that the correspondent jffs2_inode_cache will not be removed. So, it is allowed to access it without locking the inocache_lock spinlock. Index: nodelist.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/nodelist.c,v retrieving revision 1.93 retrieving revision 1.94 diff -u -r1.93 -r1.94 --- nodelist.c 27 Feb 2005 23:01:32 -0000 1.93 +++ nodelist.c 13 Apr 2005 13:22:35 -0000 1.94 @@ -489,11 +489,13 @@ void jffs2_add_ino_cache (struct jffs2_sb_info *c, struct jffs2_inode_cache *new) { struct jffs2_inode_cache **prev; - D2(printk(KERN_DEBUG "jffs2_add_ino_cache: Add %p (ino #%u)\n", new, new->ino)); - BUG_ON(!new->ino); spin_lock(&c->inocache_lock); - + if (!new->ino) + new->ino = ++c->highest_ino; + + D2(printk(KERN_DEBUG "jffs2_add_ino_cache: Add %p (ino #%u)\n", new, new->ino)); + prev = &c->inocache_list[new->ino % INOCACHE_HASHSIZE]; while ((*prev) && (*prev)->ino < new->ino) { Index: write.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs2/write.c,v retrieving revision 1.91 retrieving revision 1.92 diff -u -r1.91 -r1.92 --- write.c 1 Mar 2005 10:34:03 -0000 1.91 +++ write.c 13 Apr 2005 13:22:35 -0000 1.92 @@ -35,13 +35,12 @@ f->inocache = ic; f->inocache->nlink = 1; f->inocache->nodes = (struct jffs2_raw_node_ref *)f->inocache; - f->inocache->ino = ++c->highest_ino; f->inocache->state = INO_STATE_PRESENT; - ri->ino = cpu_to_je32(f->inocache->ino); - D1(printk(KERN_DEBUG "jffs2_do_new_inode(): Assigned ino# %d\n", f->inocache->ino)); jffs2_add_ino_cache(c, f->inocache); + D1(printk(KERN_DEBUG "jffs2_do_new_inode(): Assigned ino# %d\n", f->inocache->ino)); + ri->ino = cpu_to_je32(f->inocache->ino); ri->magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); ri->nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE); From dedekind at infradead.org Wed Apr 13 13:41:11 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.34,1.35 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv438 Modified Files: JFFS3design.tex Log Message: One more section Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.34 retrieving revision 1.35 diff -u -r1.34 -r1.35 --- JFFS3design.tex 13 Apr 2005 13:14:31 -0000 1.34 +++ JFFS3design.tex 13 Apr 2005 17:41:07 -0000 1.35 @@ -64,14 +64,14 @@ NAND flash (nandsim module). In all experiments below the whole flash comprises single partition where JFFS2 filesystem is put. -\begin{flushleft}\textbf{Experiment 1}\end{flushleft} - +\begin{description} +\item[Experiment 1] Fulfill the JFFS2 file system with a typical Linux root FS, namely \texttt{/bin}, \texttt{/sbin}, \texttt{/etc}, \texttt{/boot} and partially \texttt{/usr} from the x86 Fedora Core 2 distribution. We observed the following. -The total files number was 4372 - 719 directories and 2995 regular +The total files number was 4372 -- 719 directories and 2995 regular files (all files' nodes were made pristine). The summary size of all files was 116 MiB. The summary size of all files was 116 MiB (compression was enabled). @@ -89,7 +89,14 @@ \end{tabular} \end{center} -\begin{flushleft}\textbf{Experiment 2}\end{flushleft} +Note, that all inodes were in the inode cache in our experiments, +which isn't that typical +for the real-life system though. If no inodes were in the inode cache, +only 658/1778/12706 KiB would be consumed by the \texttt{jffs2\_node\_ref} +objects. However, opening any +file or looking up in any directory would require additional RAM. + +\item[Experiment 2] The following command on the same empty 64 MiB JFFS2 file system \begin{quote} @@ -107,7 +114,7 @@ \end{tabular} \end{center} -\begin{flushleft}\textbf{Experiment 3}\end{flushleft} +\item[Experiment 3] The following command on the same empty 64 MiB JFFS2 file system \begin{quote} @@ -125,22 +132,17 @@ \end{tabular} \end{center} -Note, that all inodes were in the inode cache in our experiments, -which isn't that typical -for the real-life system though. If no inodes were in the inode cache, -only 658/1778/12706 KiB would be consumed by the \texttt{jffs2\_node\_ref} -objects. However, opening any -file or looking up in any directory would require additional RAM. - -It is worth noting here, that in JFFS2 memory which is consumed even in -the case when no files are opened is called \emph{in-core memory}. +It is worth noting here, that in JFFS2 memory which is consumed even +when no files are opened is called \emph{in-core memory}. In-core memory mostly consists of \texttt{jffs2\_node\_ref} objects. +\end{description} -Assuming the amount consumed memory grows linearly with the flash size -(which I believe is true) we'd get the following numbers for 1GB flash -in the same experiments: +Assuming the amount of consumed memory grows linearly with the flash size +(which I believe is true) we'd have the following numbers for 1GB flash +in similar experiments. -\begin{flushleft}\textbf{Experiment 1a (imaginary)}\end{flushleft} +\begin{description} +\item[Experiment 1a (imaginary)]~ \begin{center} \begin{tabular}{ll} jffs2\_node\_ref & 10.3 MiB \\ @@ -152,7 +154,7 @@ \end{tabular} \end{center} -\begin{flushleft}\textbf{Experiment 2a (imaginary)}\end{flushleft} +\item[Experiment 2a (imaginary)]~ \begin{center} \begin{tabular}{ll} jffs2\_node\_ref & 27.8 KiB \\ @@ -163,7 +165,7 @@ \end{tabular} \end{center} -\begin{flushleft}\textbf{Experiment 3a (imaginary)}\end{flushleft} +\item[Experiment 3a (imaginary)]~ \begin{center} \begin{tabular}{ll} jffs2\_node\_ref & 198.5 KiB \\ @@ -173,9 +175,71 @@ total & 743.6 MiB \end{tabular} \end{center} +\end{description} Needless to note that this is unacceptable for embedded systems. +The following memory consumption problems are distinguished. +\begin{description} +\item[In-core RAM.] +Due to the design JFFS2 needs to refer each node associating a +small RAM object with it resulting in substantial RAM consumption. The +amount of in-core RAM depends linearly on the amount of information on +the flash. More data are put to the FS, more RAM JFFS2 takes. + +\item[Inode build RAM.] And again due to JFFS2 design built inodes (those who +are in the inode cache, which happens when, say, a file is opened or +a directory is browsed) consume a lot of RAM (). For files JFFS2 needs to +store in RAM fragment trees (\texttt{jffs2\_node\_frag} and +\texttt{jffs2\_full\_dnode} objects) and for directories -- children +lists (\texttt{jffs2\_full\_dirent} objects). Larger file you open, more +RAM is require for its fragtree. Larger directory (i.e., more directory +entries it has) you browse, more RAM is needed. + +\item[Peak RAM usage.] The JFFS2 memory consumption also depends on how data is +written. Each transaction goes directly to Flash (through the +write-buffer for page-based Flashes like NAND), i.e., there is +Write-Through cache in Linux/JFFS2. Consequently, small transactions +result in a great deal of small nodes on Flash and hence, much memory is required, +for in-core objects and fragtree. Later small nodes will be merged by +GC (maximum \texttt{PAGE\_SIZE} bytes of a files' data may be stored in +one node) and the amount of consumed memory will be much lower. But the peak +JFFS2 memory usage is very high. +\end{description} + +% +% Mount time +% +\subsection{Mount time} +The slow mount is the most prominent and upsetting JFFS2 feature. The +reason is, again, the JFFS2 design which doesn't assume any definite +structure on the Flash media but instead, makes use of one big log made +up of nodes -- the only JFFS2 on-flash data structure. This design +provides several very nice JFFS2 features as: +\begin{itemize} +\item very economical flash usage -- data usually takes as much flash +space as it actually need, without wasting much space as in case of +block devices like HDD; +\item admitting of very efficient utilizing of "on-flight" compression which +allows to fit a lot of data to Flash; +\item relatively quick read and write operations. +\end{itemize} + +But unfortunately, there are several drawbacks, for example -- slow mount. +Because of absence of any Flash structure, JFFS2 needs to scan the whole +Flash partition to identify all nodes and to build the file system map. This +takes much time, especially on large Flash partitions. + +To increase the mount speed JFFS2 performs as few work as possible during +the mount process and deffers +a considerable amount of work to GC thread. GC thread continues +working in backgroung (this process is called "checking" in JFFS2 as it +mostly check nodes' CRC checksums, albeit it also discovers obsolete +nodes building temprary fragment trees and direntry lists). And the +important thing is that during this checking proces no one could write to +the file system, only read operations aren't forbidden. This is an +additional cumberstone. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % CHECKSUM @@ -328,7 +392,7 @@ Virtual erase blocks have also disadvantages. The bigger size affects garbage collection as larger entities have to be handled, which -degrades GC efficency and performance. +degrades GC efficiency and performance. In order to keep accounting simple the number of concatenated blocks must be a power of two. @@ -338,7 +402,7 @@ The concatenation of physical blocks to virtual blocks must deal with bad blocks in the virtual block rather than treating the whole virtual block as bad as it is currently done in JFFS2. Therefor JFFS3 must treat the -physical blocks inside a virtual block seperately. This implies the +physical blocks inside a virtual block separately. This implies the clean marker write per physical block after erase and the limitation of writes to physical block boundaries. From havasi at infradead.org Thu Apr 14 05:12:12 2005 From: havasi at infradead.org (havasi@infradead.org) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 nodemgmt.c, 3.15, 3.16 scan.c, 3.16, 3.17 summary.h, 3.3, 3.4 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv3516 Modified Files: nodemgmt.c scan.c summary.h Log Message: Summary PAD fix. Index: nodemgmt.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/nodemgmt.c,v retrieving revision 3.15 retrieving revision 3.16 diff -u -r3.15 -r3.16 --- nodemgmt.c 25 Jan 2005 20:11:12 -0000 3.15 +++ nodemgmt.c 14 Apr 2005 09:12:09 -0000 3.16 @@ -184,13 +184,13 @@ if (jeb) { if ((ret=jffs3_sum_care_sum_collected(jeb))) return ret; - nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS3_SUMMARY_FRAME_SIZE; + nofree_size = PAD(sumsize + jeb->sum_collected->sum_size + JFFS3_SUMMARY_FRAME_SIZE); } DBG_SUMMARY(1,"minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n", minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize); - if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS3_SUMMARY_FRAME_SIZE > jeb->free_size)) { + if (jeb && (PAD(minsize) + PAD(jeb->sum_collected->sum_size + sumsize + JFFS3_SUMMARY_FRAME_SIZE) > jeb->free_size)) { DBG_SUMMARY(1,"generating summary for 0x%08x.\n", jeb->offset); if (jeb->sum_collected->sum_size == JFFS3_SUMMARY_NOSUM_SIZE) { sumsize = JFFS3_SUMMARY_NOSUM_SIZE; Index: scan.c =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/scan.c,v retrieving revision 3.16 retrieving revision 3.17 diff -u -r3.16 -r3.17 --- scan.c 25 Feb 2005 10:36:34 -0000 3.16 +++ scan.c 14 Apr 2005 09:12:09 -0000 3.17 @@ -668,7 +668,7 @@ #ifdef CONFIG_JFFS3_SUMMARY if (jeb->sum_collected) { - if (jeb->sum_collected->sum_size + JFFS3_SUMMARY_FRAME_SIZE > jeb->free_size) { + if (PAD(jeb->sum_collected->sum_size + JFFS3_SUMMARY_FRAME_SIZE) > jeb->free_size) { DBG_SUMMARY(1,"There is not enough space for " "summary information, freeing up summary info!\n"); jffs3_sum_clean_collected(jeb); Index: summary.h =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/summary.h,v retrieving revision 3.3 retrieving revision 3.4 diff -u -r3.3 -r3.4 --- summary.h 5 Jan 2005 16:19:00 -0000 3.3 +++ summary.h 14 Apr 2005 09:12:09 -0000 3.4 @@ -106,7 +106,7 @@ jint32_t magic; }; -#define JFFS3_SUMMARY_FRAME_SIZE (sizeof(struct jffs3_summary_node)+sizeof(struct jffs3_sum_marker)+3) +#define JFFS3_SUMMARY_FRAME_SIZE (sizeof(struct jffs3_summary_node) + sizeof(struct jffs3_sum_marker)) #if !(defined(SUM_TOOL) || defined(JFFS2DUMP)) From havasi at infradead.org Thu Apr 14 05:13:36 2005 From: havasi at infradead.org (havasi@infradead.org) Date: Mon May 15 21:51:47 2006 Subject: mtd/util mkfs.jffs2.c,1.44,1.45 Message-ID: Update of /home/cvs/mtd/util In directory phoenix.infradead.org:/tmp/cvs-serv3549 Modified Files: mkfs.jffs2.c Log Message: option name fix Index: mkfs.jffs2.c =================================================================== RCS file: /home/cvs/mtd/util/mkfs.jffs2.c,v retrieving revision 1.44 retrieving revision 1.45 diff -u -r1.44 -r1.45 --- mkfs.jffs2.c 17 Feb 2005 14:55:07 -0000 1.44 +++ mkfs.jffs2.c 14 Apr 2005 09:13:34 -0000 1.45 @@ -1164,7 +1164,7 @@ {"disable-compressor", 1, NULL, 'x'}, {"test-compression", 0, NULL, 't'}, {"compressor-priority", 1, NULL, 'y'}, - {"input", 1, NULL, 'i'}, + {"incremental", 1, NULL, 'i'}, {NULL, 0, NULL, 0} }; From dedekind at infradead.org Thu Apr 14 08:43:04 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.35,1.36 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv4189 Modified Files: JFFS3design.tex Log Message: Add inode build chapter, fixes Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.35 retrieving revision 1.36 diff -u -r1.35 -r1.36 --- JFFS3design.tex 13 Apr 2005 17:41:07 -0000 1.35 +++ JFFS3design.tex 14 Apr 2005 12:43:00 -0000 1.36 @@ -222,7 +222,8 @@ block devices like HDD; \item admitting of very efficient utilizing of "on-flight" compression which allows to fit a lot of data to Flash; -\item relatively quick read and write operations. +\item relatively quick read and write operations; +\item natural unclean reboot robustness; \end{itemize} But unfortunately, there are several drawbacks, for example -- slow mount. @@ -231,14 +232,52 @@ takes much time, especially on large Flash partitions. To increase the mount speed JFFS2 performs as few work as possible during -the mount process and deffers +the mount process and defers a considerable amount of work to GC thread. GC thread continues -working in backgroung (this process is called "checking" in JFFS2 as it +working in background (this process is called "checking" in JFFS2 as it mostly check nodes' CRC checksums, albeit it also discovers obsolete -nodes building temprary fragment trees and direntry lists). And the -important thing is that during this checking proces no one could write to +nodes building temporary fragment trees and direntry lists). And the +important thing is that during this checking process GC can not proceed and in +many cases no one could write to the file system, only read operations aren't forbidden. This is an -additional cumberstone. +additional cumber-stone. + +Thus, there are following mount problems in JFFS2: +\begin{description} +\item[Slow mount] To mount the file system JFFS2 scans the whole partition in +order to recognize nodes, build per-block and per-inode nodes lists, etc. This +is slow. +\item[Unpleasant checking.] Just after mount JFFS2 checks all the +inodes which results in flash re-reading, eats a lot of CPU cycles and may block +writers. +\end{description} + +TODO: show some real numbers here. + +The first problem may be solved by a patch referred to as the "summary patch" and +accessible at \url{http://www.inf.u-szeged.hu/jffs2/mount.php}. + +% +% Inode build +% +\subsection{Inode build} +One more JFFS2 problem is the slow inode build (\texttt{iget()} VFS call) +implementation for both regular files and directories. Users typically start +feeling uncomfortable starting from file sizes of about 32 MiB (this is highly +dependent on how the file was written, i.e., how many nodes correspond to the +file). + +Thus, there is one more problem with JFFS2: +\begin{description} +\item[Slow inode build.] The inode build process implies that JFFS2 walks +through all the inode's nodes +(which are data nodes for regular file and direntries for directory), +reads the nodes, checks their CRC and builds fragtree/dirent list. And the inode +build time is the linear function of the nodes number. +\end{description} + +TODO: provide some real numbers here + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Thu Apr 14 13:25:27 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.36,1.37 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv5224 Modified Files: JFFS3design.tex Log Message: Conclusion chapter. Fixes. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.36 retrieving revision 1.37 diff -u -r1.36 -r1.37 --- JFFS3design.tex 14 Apr 2005 12:43:00 -0000 1.36 +++ JFFS3design.tex 14 Apr 2005 17:25:23 -0000 1.37 @@ -60,8 +60,8 @@ % Memory consumption % \subsection{Memory consumption} -Let's make experiments using an x86 PC host with 64MiB simulated -NAND flash (nandsim module). In all experiments below the whole +Let's make experiments using an x86 PC host with 64~MiB simulated +NAND flash (\emph{nandsim} simulator module). In all the experiments below the whole flash comprises single partition where JFFS2 filesystem is put. \begin{description} @@ -72,8 +72,9 @@ We observed the following. The total files number was 4372 -- 719 directories and 2995 regular -files (all files' nodes were made pristine). The summary size of all files was 116 MiB. -The summary size of all files was 116 MiB (compression was enabled). +files (all the files' nodes were made pristine, i.e., +\texttt{PAGE\_SIZE} bytes in length). +The summary size of the all files was 116~MiB (compression was enabled). The memory consumed by JFFS2 was distributed over its internal objects as follows. @@ -89,15 +90,15 @@ \end{tabular} \end{center} -Note, that all inodes were in the inode cache in our experiments, +Note, that all the inodes were in the Linux Inode Cache in our experiment, which isn't that typical -for the real-life system though. If no inodes were in the inode cache, -only 658/1778/12706 KiB would be consumed by the \texttt{jffs2\_node\_ref} +for the real-life system though. If no inodes were in the Inode Cache, +only 658~KiB would be consumed by the \texttt{jffs2\_node\_ref} objects. However, opening any file or looking up in any directory would require additional RAM. \item[Experiment 2] -The following command on the same empty 64 MiB JFFS2 file +The following command on the same empty 64~MiB JFFS2 file system \begin{quote} \texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=512} @@ -115,7 +116,7 @@ \end{center} \item[Experiment 3] -The following command on the same empty 64 MiB JFFS2 file +The following command on the same empty 64~MiB JFFS2 file system \begin{quote} \texttt{dd if=/dev/urandom of=/mnt/jffs\_mnt/file bs=10} @@ -133,12 +134,12 @@ \end{center} It is worth noting here, that in JFFS2 memory which is consumed even -when no files are opened is called \emph{in-core memory}. -In-core memory mostly consists of \texttt{jffs2\_node\_ref} objects. +when no files are opened is called \emph{\mbox{in-core} memory}. +\mbox{In-core} memory mostly consists of \texttt{jffs2\_node\_ref} objects. \end{description} Assuming the amount of consumed memory grows linearly with the flash size -(which I believe is true) we'd have the following numbers for 1GB flash +(which I believe is true) we'd have the following numbers for an 1GB flash in similar experiments. \begin{description} @@ -184,24 +185,24 @@ \item[In-core RAM.] Due to the design JFFS2 needs to refer each node associating a small RAM object with it resulting in substantial RAM consumption. The -amount of in-core RAM depends linearly on the amount of information on +amount of \mbox{in-core} RAM depends linearly on the amount of information on the flash. More data are put to the FS, more RAM JFFS2 takes. -\item[Inode build RAM.] And again due to JFFS2 design built inodes (those who -are in the inode cache, which happens when, say, a file is opened or -a directory is browsed) consume a lot of RAM (). For files JFFS2 needs to +\item[Inode build RAM.] And again, due to JFFS2 design built inodes (those who +are in the Inode Cache, which happens when, say, when a file is opened or +a directory is browsed) consume a lot of RAM. For regular files JFFS2 needs to store in RAM fragment trees (\texttt{jffs2\_node\_frag} and \texttt{jffs2\_full\_dnode} objects) and for directories -- children -lists (\texttt{jffs2\_full\_dirent} objects). Larger file you open, more +lists (\texttt{jffs2\_full\_dirent} objects). Larger file is opened, more RAM is require for its fragtree. Larger directory (i.e., more directory -entries it has) you browse, more RAM is needed. +entries in it) is browsed, more RAM is needed. \item[Peak RAM usage.] The JFFS2 memory consumption also depends on how data is written. Each transaction goes directly to Flash (through the -write-buffer for page-based Flashes like NAND), i.e., there is -Write-Through cache in Linux/JFFS2. Consequently, small transactions +\mbox{write-buffer} for \mbox{page-based} Flashes like NAND), i.e., there is +\mbox{Write-Through} cache in Linux/JFFS2. Consequently, small transactions result in a great deal of small nodes on Flash and hence, much memory is required, -for in-core objects and fragtree. Later small nodes will be merged by +for \mbox{in-core} objects and fragtree. Later small nodes will be merged by GC (maximum \texttt{PAGE\_SIZE} bytes of a files' data may be stored in one node) and the amount of consumed memory will be much lower. But the peak JFFS2 memory usage is very high. @@ -214,18 +215,7 @@ The slow mount is the most prominent and upsetting JFFS2 feature. The reason is, again, the JFFS2 design which doesn't assume any definite structure on the Flash media but instead, makes use of one big log made -up of nodes -- the only JFFS2 on-flash data structure. This design -provides several very nice JFFS2 features as: -\begin{itemize} -\item very economical flash usage -- data usually takes as much flash -space as it actually need, without wasting much space as in case of -block devices like HDD; -\item admitting of very efficient utilizing of "on-flight" compression which -allows to fit a lot of data to Flash; -\item relatively quick read and write operations; -\item natural unclean reboot robustness; -\end{itemize} - +up of nodes -- the only JFFS2 \mbox{on-flash} data structure. But unfortunately, there are several drawbacks, for example -- slow mount. Because of absence of any Flash structure, JFFS2 needs to scan the whole Flash partition to identify all nodes and to build the file system map. This @@ -233,29 +223,28 @@ To increase the mount speed JFFS2 performs as few work as possible during the mount process and defers -a considerable amount of work to GC thread. GC thread continues -working in background (this process is called "checking" in JFFS2 as it +a considerable amount of work to the GC thread. The GC thread +works in background (this process is called "checking" in JFFS2 as it mostly check nodes' CRC checksums, albeit it also discovers obsolete nodes building temporary fragment trees and direntry lists). And the important thing is that during this checking process GC can not proceed and in many cases no one could write to the file system, only read operations aren't forbidden. This is an -additional cumber-stone. +additional \mbox{cumber-stone}. Thus, there are following mount problems in JFFS2: \begin{description} \item[Slow mount] To mount the file system JFFS2 scans the whole partition in -order to recognize nodes, build per-block and per-inode nodes lists, etc. This -is slow. +order to recognize nodes, build \mbox{per-block} and \mbox{per-inode} nodes lists, etc. \item[Unpleasant checking.] Just after mount JFFS2 checks all the -inodes which results in flash re-reading, eats a lot of CPU cycles and may block -writers. +inodes which results in flash \mbox{re-reading}, eats a lot of CPU cycles and may block +writers for a considerable time interval. \end{description} TODO: show some real numbers here. -The first problem may be solved by a patch referred to as the "summary patch" and -accessible at \url{http://www.inf.u-szeged.hu/jffs2/mount.php}. +The first problem may be solved by a patch referred to as a "summary patch" and +accessible at \url{http://www.inf.u-szeged.hu/jffs2/}. % % Inode build @@ -263,7 +252,7 @@ \subsection{Inode build} One more JFFS2 problem is the slow inode build (\texttt{iget()} VFS call) implementation for both regular files and directories. Users typically start -feeling uncomfortable starting from file sizes of about 32 MiB (this is highly +feeling uncomfortable starting from file sizes of about 32~MiB (this is highly dependent on how the file was written, i.e., how many nodes correspond to the file). @@ -278,6 +267,26 @@ TODO: provide some real numbers here +\subsection{Conclusion} +Historically, JFFS2 was designed for small NOR flash chips and didn't even +support NAND, whose support was added later. The JFFS2 is perfectly good for +small flashes, and has such nice characteristics as: + +\begin{itemize} +\item very economical flash usage -- data usually takes as much flash +space as it actually need, without wasting much space as in case of +"traditional" file systems for block devices; +\item admitting of very efficient utilizing of "on-flight" compression which +allows to fit a big deal of data to Flash; +\item quick read and write operations; +\item natural unclean reboot robustness; +\item good wear-leveling. +\end{itemize} + +But unfortunately, it has one big drawback - bad scalability. Namely, mount +time, memory consumption and inode build time are liner functions of the +amount of the stored data. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Tue Apr 19 11:51:44 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.37,1.38 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv12071 Modified Files: JFFS3design.tex Log Message: Add a paragraph. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.37 retrieving revision 1.38 diff -u -r1.37 -r1.38 --- JFFS3design.tex 14 Apr 2005 17:25:23 -0000 1.37 +++ JFFS3design.tex 19 Apr 2005 15:51:40 -0000 1.38 @@ -287,6 +287,25 @@ time, memory consumption and inode build time are liner functions of the amount of the stored data. +\section{JFFS3 goals and requirements} +The following is somewhat unstructured list of JFFS2 goals and requirements. + +\begin{enumerate} +\item Fully POSIX-compatible file system. +\item The main OS is Linux (TODO: nonetheless portable?). +\item Good scalability. +\item Preserve the possibility to efficiently use compression (like in JFFS2). +\item Provide good wear-leveling. +\item Tolerant to unclean reboots. +\item Fast mounting. +\item Reasonably low memory consumption. +\item Fast enough read/write operations. +\item Support different Flash types (NOR, NAND, etc). +\item Support xattr. +\end{enumerate} + +Of course there are a number of tradeoffs and we should find a balanced compromise +solution, %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Wed Apr 20 10:23:54 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.38,1.39 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv16029 Modified Files: JFFS3design.tex Log Message: Add CP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.38 retrieving revision 1.39 diff -u -r1.38 -r1.39 --- JFFS3design.tex 19 Apr 2005 15:51:40 -0000 1.38 +++ JFFS3design.tex 20 Apr 2005 14:23:51 -0000 1.39 @@ -234,7 +234,7 @@ Thus, there are following mount problems in JFFS2: \begin{description} -\item[Slow mount] To mount the file system JFFS2 scans the whole partition in +\item[Slow mount.] To mount the file system JFFS2 scans the whole partition in order to recognize nodes, build \mbox{per-block} and \mbox{per-inode} nodes lists, etc. \item[Unpleasant checking.] Just after mount JFFS2 checks all the inodes which results in flash \mbox{re-reading}, eats a lot of CPU cycles and may block @@ -287,6 +287,11 @@ time, memory consumption and inode build time are liner functions of the amount of the stored data. +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% +% JFFS3 GOALS AND REQUIREMENTS +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{JFFS3 goals and requirements} The following is somewhat unstructured list of JFFS2 goals and requirements. @@ -309,6 +314,73 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % +% CHECKPOINTS +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\section{Checkpoints} +A checkpoint (CP) is new node type which serves to accumulate all +information about an inode which is required to efficiently read and +write to files create/remove direntries in directories. The following +types of checkpoints are defined: + +\begin{itemize} +\item data checkpoints which are associated with regular files; +\item directory checkpoints which are associated with directories; +\end{itemize} + +% +% Data checkpoints +% +\subsection{Data checkpoints} +Data checkpoints (DCP): +\begin{itemize} +\item are associated with regular files; +\item refer all the valid nodes of a regular file inode; +\item allow not to keep in-core references to all the inode data nodes, +but only references to the inode data checkpoints; +\item play the role of JFFS2 fragtrees, i.e., allow to quickly locate +positions of data nodes for any given file data range. +\end{itemize} + +Each regular file inode is associated with one or more data +checkpoints. Each DCP corresponds to a fixed inode data range of size $R$. +I.e., if the size of the regular file is $< R$, it has +only one associated DCP node. If the size of the file is $> {R}$ but +$< {2R}$ then there will be two associated DCP and so forth. +Obviously, $R$ value is multiple of \texttt{PAGE\_SIZE}. + +Each DCP node carries the following information: +\begin{itemize} +\item index $I$ +which defines the DCP range (the file range described by DCP is +${\lbrack}I{\cdot}R, I{\cdot}(R + 1)\rbrack$); + +\item version, in order to distinguish valid and obsolete DCP +nodes; + +\item the list of data nodes belonging to the DCP range; the list is +sorted by the data node offsets, i.e., is effectively the fragtree of +the DCP range. +\end{itemize} + +The above mentioned list is essentially an array of objects, containing +the following: +\begin{itemize} +\item the data node range; +\item the data node physical address of Flash. +\end{itemize} + +Thus, larger files have more associated DCP nodes. It stands for reason +that when file is changed or GC moves any node of the file, the +corresponding DCP node becomes obsolete and ought to be rewritten. But +only those data checkpoints are rewritten that correspond to the changed +file range. + +With the help of DCP JFFS3 need only keep in-core an array (or more +precisely, a list of arrays) of DCP node references of files. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% % CHECKSUM % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% From dedekind at infradead.org Wed Apr 20 12:10:43 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:47 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.39,1.40 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv16314 Modified Files: JFFS3design.tex Log Message: Updates. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.39 retrieving revision 1.40 diff -u -r1.39 -r1.40 --- JFFS3design.tex 20 Apr 2005 14:23:51 -0000 1.39 +++ JFFS3design.tex 20 Apr 2005 16:10:38 -0000 1.40 @@ -310,7 +310,7 @@ \end{enumerate} Of course there are a number of tradeoffs and we should find a balanced compromise -solution, +solution. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % @@ -325,7 +325,7 @@ \begin{itemize} \item data checkpoints which are associated with regular files; -\item directory checkpoints which are associated with directories; +\item directory checkpoints which are associated with directories. \end{itemize} % @@ -379,6 +379,29 @@ With the help of DCP JFFS3 need only keep in-core an array (or more precisely, a list of arrays) of DCP node references of files. +The precise value of $R$ depends on the memory consumption requirements. +The following table illustrates the dependency of the consumed memory and $R$ +for different file sizes (we assume each DCP reference takes 4 bytes). + +\begin{center} +\begin{tabular}{lll} +$R$ & File size & RAM required\\ +\hline +512 KiB & 128 MiB & 1 MiB\\ +1 MiB & 128 MiB & 0.5 MiB\\ +4 MiB & 4 GiB & 4 MiB\\ +8 MiB & 4 GiB & 1 MiB\\ +\end{tabular} +\end{center} + +The $R$ value may be determined from the flash size and the desired RAM +consumption limit and may be configurable. + +Since most files on the file system are small in size, we may facilitate +faster changing of small files by making $R$ smaller for the beginning +of files and and larger for the rest of files. I.e, 256~KiB for file +ranges before 2~MiB and 8~MiB for ranges after 2~MiB. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % CHECKSUM From joern at infradead.org Wed Apr 20 23:42:15 2005 From: joern at infradead.org (joern@infradead.org) Date: Mon May 15 21:51:48 2006 Subject: mtd/drivers/mtd/devices mtdram.c,1.36,1.37 Message-ID: Update of /home/cvs/mtd/drivers/mtd/devices In directory phoenix.infradead.org:/home/joern/mtd/drivers/mtd/devices Modified Files: mtdram.c Log Message: Quick cleanup of the driver: o Lindent o Removal of slram/phram functionality o Removal of most #ifdefs Index: mtdram.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/devices/mtdram.c,v retrieving revision 1.36 retrieving revision 1.37 diff -u -r1.36 -r1.37 --- mtdram.c 17 Mar 2005 19:44:27 -0000 1.36 +++ mtdram.c 21 Apr 2005 03:42:11 -0000 1.37 @@ -4,11 +4,10 @@ * Author: Alexander Larsson * * Copyright (c) 1999 Alexander Larsson + * Copyright (c) 2005 Joern Engel * * This code is GPL - * */ - #include #include #include @@ -18,213 +17,140 @@ #include #include -#ifndef CONFIG_MTDRAM_ABS_POS - #define CONFIG_MTDRAM_ABS_POS 0 -#endif - -#if CONFIG_MTDRAM_ABS_POS > 0 - #include -#endif - -#ifdef MODULE static unsigned long total_size = CONFIG_MTDRAM_TOTAL_SIZE; static unsigned long erase_size = CONFIG_MTDRAM_ERASE_SIZE; -module_param(total_size,ulong,0); -MODULE_PARM_DESC(total_size, "Total device size in KiB"); -module_param(erase_size,ulong,0); -MODULE_PARM_DESC(erase_size, "Device erase block size in KiB"); #define MTDRAM_TOTAL_SIZE (total_size * 1024) #define MTDRAM_ERASE_SIZE (erase_size * 1024) -#else -#define MTDRAM_TOTAL_SIZE (CONFIG_MTDRAM_TOTAL_SIZE * 1024) -#define MTDRAM_ERASE_SIZE (CONFIG_MTDRAM_ERASE_SIZE * 1024) -#endif +#ifdef MODULE +module_param(total_size, ulong, 0); +MODULE_PARM_DESC(total_size, "Total device size in KiB"); +module_param(erase_size, ulong, 0); +MODULE_PARM_DESC(erase_size, "Device erase block size in KiB"); +#endif // We could store these in the mtd structure, but we only support 1 device.. static struct mtd_info *mtd_info; - -static int -ram_erase(struct mtd_info *mtd, struct erase_info *instr) +static int ram_erase(struct mtd_info *mtd, struct erase_info *instr) { - DEBUG(MTD_DEBUG_LEVEL2, "ram_erase(pos:%ld, len:%ld)\n", (long)instr->addr, (long)instr->len); - if (instr->addr + instr->len > mtd->size) { - DEBUG(MTD_DEBUG_LEVEL1, "ram_erase() out of bounds (%ld > %ld)\n", (long)(instr->addr + instr->len), (long)mtd->size); - return -EINVAL; - } - - memset((char *)mtd->priv + instr->addr, 0xff, instr->len); - - instr->state = MTD_ERASE_DONE; - mtd_erase_callback(instr); + if (instr->addr + instr->len > mtd->size) + return -EINVAL; + + memset((char *)mtd->priv + instr->addr, 0xff, instr->len); - return 0; + instr->state = MTD_ERASE_DONE; + mtd_erase_callback(instr); + + return 0; } -static int ram_point (struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char **mtdbuf) +static int ram_point(struct mtd_info *mtd, loff_t from, size_t len, + size_t *retlen, u_char **mtdbuf) { - if (from + len > mtd->size) - return -EINVAL; - - *mtdbuf = mtd->priv + from; - *retlen = len; - return 0; + if (from + len > mtd->size) + return -EINVAL; + + *mtdbuf = mtd->priv + from; + *retlen = len; + return 0; } -static void ram_unpoint (struct mtd_info *mtd, u_char *addr, loff_t from, - size_t len) +static void ram_unpoint(struct mtd_info *mtd, u_char * addr, loff_t from, + size_t len) { - DEBUG(MTD_DEBUG_LEVEL2, "ram_unpoint\n"); } static int ram_read(struct mtd_info *mtd, loff_t from, size_t len, - size_t *retlen, u_char *buf) + size_t *retlen, u_char *buf) { - DEBUG(MTD_DEBUG_LEVEL2, "ram_read(pos:%ld, len:%ld)\n", (long)from, (long)len); - if (from + len > mtd->size) { - DEBUG(MTD_DEBUG_LEVEL1, "ram_read() out of bounds (%ld > %ld)\n", (long)(from + len), (long)mtd->size); - return -EINVAL; - } + if (from + len > mtd->size) + return -EINVAL; - memcpy(buf, mtd->priv + from, len); + memcpy(buf, mtd->priv + from, len); - *retlen=len; - return 0; + *retlen = len; + return 0; } static int ram_write(struct mtd_info *mtd, loff_t to, size_t len, - size_t *retlen, const u_char *buf) + size_t *retlen, const u_char *buf) { - DEBUG(MTD_DEBUG_LEVEL2, "ram_write(pos:%ld, len:%ld)\n", (long)to, (long)len); - if (to + len > mtd->size) { - DEBUG(MTD_DEBUG_LEVEL1, "ram_write() out of bounds (%ld > %ld)\n", (long)(to + len), (long)mtd->size); - return -EINVAL; - } + if (to + len > mtd->size) + return -EINVAL; - memcpy ((char *)mtd->priv + to, buf, len); + memcpy((char *)mtd->priv + to, buf, len); - *retlen=len; - return 0; + *retlen = len; + return 0; } static void __exit cleanup_mtdram(void) { - if (mtd_info) { - del_mtd_device(mtd_info); -#if CONFIG_MTDRAM_TOTAL_SIZE > 0 - if (mtd_info->priv) -#if CONFIG_MTDRAM_ABS_POS > 0 - iounmap(mtd_info->priv); -#else - vfree(mtd_info->priv); -#endif -#endif - kfree(mtd_info); - } -} - -int mtdram_init_device(struct mtd_info *mtd, void *mapped_address, - unsigned long size, char *name) -{ - memset(mtd, 0, sizeof(*mtd)); + if (mtd_info) { + del_mtd_device(mtd_info); + if (mtd_info->priv) + vfree(mtd_info->priv); + kfree(mtd_info); + } +} + +int mtdram_init_device(struct mtd_info *mtd, void *mapped_address, + unsigned long size, char *name) +{ + memset(mtd, 0, sizeof(*mtd)); + + /* Setup the MTD structure */ + mtd->name = name; + mtd->type = MTD_RAM; + mtd->flags = MTD_CAP_RAM; + mtd->size = size; + mtd->erasesize = MTDRAM_ERASE_SIZE; + mtd->priv = mapped_address; + + mtd->owner = THIS_MODULE; + mtd->erase = ram_erase; + mtd->point = ram_point; + mtd->unpoint = ram_unpoint; + mtd->read = ram_read; + mtd->write = ram_write; + + if (add_mtd_device(mtd)) { + return -EIO; + } - /* Setup the MTD structure */ - mtd->name = name; - mtd->type = MTD_RAM; - mtd->flags = MTD_CAP_RAM; - mtd->size = size; - mtd->erasesize = MTDRAM_ERASE_SIZE; - mtd->priv = mapped_address; - - mtd->owner = THIS_MODULE; - mtd->erase = ram_erase; - mtd->point = ram_point; - mtd->unpoint = ram_unpoint; - mtd->read = ram_read; - mtd->write = ram_write; - - if (add_mtd_device(mtd)) { - return -EIO; - } - - return 0; + return 0; } -#if CONFIG_MTDRAM_TOTAL_SIZE > 0 -#if CONFIG_MTDRAM_ABS_POS > 0 static int __init init_mtdram(void) { - void *addr; - int err; - /* Allocate some memory */ - mtd_info = kmalloc(sizeof(struct mtd_info), GFP_KERNEL); - if (!mtd_info) - return -ENOMEM; - - addr = ioremap(CONFIG_MTDRAM_ABS_POS, MTDRAM_TOTAL_SIZE); - if (!addr) { - DEBUG(MTD_DEBUG_LEVEL1, - "Failed to ioremap) memory region of size %ld at ABS_POS:%ld\n", - (long)MTDRAM_TOTAL_SIZE, (long)CONFIG_MTDRAM_ABS_POS); - kfree(mtd_info); - mtd_info = NULL; - return -ENOMEM; - } - err = mtdram_init_device(mtd_info, addr, - MTDRAM_TOTAL_SIZE, "mtdram test device"); - if (err) - { - iounmap(addr); - kfree(mtd_info); - mtd_info = NULL; - return err; - } - memset(mtd_info->priv, 0xff, MTDRAM_TOTAL_SIZE); - return err; -} + void *addr; + int err; -#else /* CONFIG_MTDRAM_ABS_POS > 0 */ + if (!total_size) + return -EINVAL; -static int __init init_mtdram(void) -{ - void *addr; - int err; - /* Allocate some memory */ - mtd_info = kmalloc(sizeof(struct mtd_info), GFP_KERNEL); - if (!mtd_info) - return -ENOMEM; - - addr = vmalloc(MTDRAM_TOTAL_SIZE); - if (!addr) { - DEBUG(MTD_DEBUG_LEVEL1, - "Failed to vmalloc memory region of size %ld\n", - (long)MTDRAM_TOTAL_SIZE); - kfree(mtd_info); - mtd_info = NULL; - return -ENOMEM; - } - err = mtdram_init_device(mtd_info, addr, - MTDRAM_TOTAL_SIZE, "mtdram test device"); - if (err) - { - vfree(addr); - kfree(mtd_info); - mtd_info = NULL; - return err; - } - memset(mtd_info->priv, 0xff, MTDRAM_TOTAL_SIZE); - return err; + /* Allocate some memory */ + mtd_info = kmalloc(sizeof(struct mtd_info), GFP_KERNEL); + if (!mtd_info) + return -ENOMEM; + + addr = vmalloc(MTDRAM_TOTAL_SIZE); + if (!addr) { + kfree(mtd_info); + mtd_info = NULL; + return -ENOMEM; + } + err = mtdram_init_device(mtd_info, addr, MTDRAM_TOTAL_SIZE, "mtdram test device"); + if (err) { + vfree(addr); + kfree(mtd_info); + mtd_info = NULL; + return err; + } + memset(mtd_info->priv, 0xff, MTDRAM_TOTAL_SIZE); + return err; } -#endif /* !(CONFIG_MTDRAM_ABS_POS > 0) */ - -#else /* CONFIG_MTDRAM_TOTAL_SIZE > 0 */ - -static int __init init_mtdram(void) -{ - return 0; -} -#endif /* !(CONFIG_MTDRAM_TOTAL_SIZE > 0) */ module_init(init_mtdram); module_exit(cleanup_mtdram); @@ -232,4 +158,3 @@ MODULE_LICENSE("GPL"); MODULE_AUTHOR("Alexander Larsson "); MODULE_DESCRIPTION("Simulated MTD driver for testing"); - From dbrown at infradead.org Thu Apr 21 11:25:06 2005 From: dbrown at infradead.org (dbrown@infradead.org) Date: Mon May 15 21:51:48 2006 Subject: mtd/docboot doc_bootstub.S,1.6,1.7 Message-ID: Update of /home/cvs/mtd/docboot In directory phoenix.infradead.org:/tmp/cvs-serv21629 Modified Files: doc_bootstub.S Log Message: Detect more than 16M of high memory. Also add some cleanups and error checking. Index: doc_bootstub.S =================================================================== RCS file: /home/cvs/mtd/docboot/doc_bootstub.S,v retrieving revision 1.6 retrieving revision 1.7 diff -u -r1.6 -r1.7 --- doc_bootstub.S 5 Apr 2005 19:36:21 -0000 1.6 +++ doc_bootstub.S 21 Apr 2005 15:25:03 -0000 1.7 @@ -69,7 +69,9 @@ 2) Copy the rest of the kernel (the protected-mode part) from the DOC into high memory starting at 0x100000. BIOS int15h function 87h is used to perform the low-to-high copy. - 3) Patch some variables in the real-mode kernel, that a Linux bootloader is + 3) Copy the initrd (if there is one) to the top of high memory using + int15/87h. + 4) Patch some variables in the real-mode kernel, that a Linux bootloader is supposed to set. Then jump to the real-mode kernel. */ @@ -80,10 +82,10 @@ cld - movw setup_seg, %es + pushw %cs + movw doc_seg, %ds movw $BXREG, %bx - movw $SIREG, %si /* Enable the DiskOnChip ASIC */ #ifdef MILPLUS @@ -101,20 +103,17 @@ call doc_cmd #endif + popw %ds /* now ds == cs */ xorw %di, %di xorw %dx, %dx read_setup_sects: call doc_readpage - decw %cs:low_sects + decw low_sects jnz read_setup_sects #ifdef DEBUG_BUILD /* Print the kernel version string. */ - pushw %ds - pushw %si pushw %bx - pushw %cs - popw %ds MSG(crlf_string) pushw %es popw %ds @@ -125,33 +124,27 @@ popw %ds MSG(kernel_string) popw %bx - popw %si - popw %ds #endif - movw $0x8000, %di /* Go to 32k from setup base */ - call read_high_sects /* copy the kernel sectors */ + pushw %es - pushw %cs - popw %ds + movw $0x8000, %di /* Go to 32k from setup base */ + call read_high_sects /* copy the kernel sectors */ + + /* At this point, cx=0 */ movw initrd_sects, %ax orw %ax, high_sects /* high_sects was previously 0 */ jz skip_initrd + movb %cl, gdt_dst_mid movw initrd_start, %ax - movw %ax, gdt_dst_mid + movb %al, gdt_dst_hi + movb %ah, gdt_dst_vhi - pushw %si pushw %bx MSG(initrd_string) popw %bx - popw %si - - movw doc_seg, %ds - call read_high_sects /* copy the initrd sectors */ - - pushw %cs - popw %ds + call read_high_sects /* copy the initrd sectors */ skip_initrd: @@ -164,13 +157,12 @@ movl initrd_bytes, %eax - pushw %es popw %ds /* Now write parameters into setup segment */ orl %eax, ramdisk_size /* ramdisk_size should be 0 before this */ jz skip_initrd2 movw %cs:initrd_start, %ax - movw %ax, ramdisk_image + 1 + movw %ax, ramdisk_image + 2 /* initrd_start is 64k aligned */ skip_initrd2: /* Compute the physical address of the commandline and @@ -206,23 +198,30 @@ /* read_high_sects: Read pages from DOC into high memory. Each page is first loaded into low memory using doc_readpage, then copied to high memory using int 15h function 87h. - We read cs:high_sects pages (leaving cs:high_sects = 0 when + We read high_sects pages (leaving high_sects = 0 when we're done.) */ read_high_sects: call doc_readpage - movw %cx, %di /* re-use the same low memory buffer */ - pushw %es + movw %ax, %di /* re-use the same low memory buffer */ pushw %cs popw %es - pushw %si movw $gdt, %si - movw $0x200, %cx + movw $0x100, %cx movb $0x87, %ah + stc int $0x15 - popw %si - popw %es - addw $2, %cs:gdt_dst_mid - decw %cs:high_sects + jc highcopy_fail + test %ah, %ah + jz highcopy_ok +highcopy_fail: + pushw %cs + popw %ds + MSG(copyfail_string) +1: jmp 1b /* hang. */ +highcopy_ok: + addw $2, gdt_dst_mid + adcb $0, gdt_dst_vhi + decw high_sects jnz read_high_sects ret /***************************************************************************/ @@ -248,6 +247,10 @@ movw %dx, %ax call phword #endif + /* cs must == ds at this point */ + movw $SIREG, %si + movw setup_seg, %es + movw doc_seg, %ds #ifdef MILPLUS /* Flash command: Reset */ movb $NAND_CMD_RESET, %al @@ -299,11 +302,17 @@ movsb decw %si loop rploop + pushw %cs + popw %ds /* restore ds == cs */ incw %dx + jnz dx_ok + MSG(nodata_string) +1: jmp 1b /* hang. */ +dx_ok: cmpw $0xb1db, %es:-2(%di) jne nextpage subw $8, %di - popw %cx /* di = old di + 512. Return original di in cx */ + popw %ax /* di = old di + 512. Return original di in ax. cx=0. */ ret /* doc_cmd: Send a command to the flash chip */ @@ -335,7 +344,7 @@ DOC_DELAY4 /* Write the actual command */ SLOWIO_WRITE(%al) - movb %al, SI_CDSN_IO + movb %al, BX_CDSN_IO /* Can't use SI_CDSN_IO here! */ /* fall through... */ /* doc_wait: Wait for the DiskOnChip to be ready */ @@ -428,14 +437,16 @@ gdt_src_lo: .byte 0 gdt_src_mid: .byte 0x80 gdt_src_hi: .byte 0 -gdt_src_perm: .byte 0x93 - .word 0 +gdt_src_flags1: .byte 0x93 +gdt_src_flags2: .byte 0 +gdt_src_vhi: .byte 0 gdt_dst_limit: .word 0xffff gdt_dst_lo: .byte 0 gdt_dst_mid: .byte 0 gdt_dst_hi: .byte 0x10 -gdt_dst_perm: .byte 0x93 - .word 0 +gdt_dst_flags1: .byte 0x93 +gdt_dst_flags2: .byte 0 +gdt_dst_vhi: .byte 0 .skip 0x10 #ifdef DEBUG_BUILD @@ -448,6 +459,9 @@ done_string: .string "done.\n\rCommandline: " #endif +copyfail_string:.string "COPY TO HIGH MEMORY FAILED" +failed_string = . - 7 +nodata_string: .string "CANNOT FIND IMAGE ON DEVICE" boot_string: .string "\n\rBooting!\n\r" crlf_string = (. - 3) @@ -506,6 +520,8 @@ cmpsb je skip_install + pushw %dx /* push 0 */ + MSG(installer_string) #ifdef DEBUG_BUILD @@ -528,7 +544,24 @@ shrw $4, %ax addw %ax, gdt_src_mid - + /* Memory-size code below adapted from Syslinux */ + movw $(0x4000-0x400), %bp /* 15M in 1k chunks */ +#ifdef DEBUG_BUILD + MSG(int15e801_string) +#endif + stc + movw $0xe801, %ax + int $0x15 + jc e801_fail + cmpw %bp, %ax + jne e801_fail + movw %bx, %ax /* now ax = amount of mem -16M, in 64k blocks */ +#ifdef DEBUG_BUILD + call phword +#endif + incb %ah /* Add 16M (in 64k blocks) back in. */ + jmp got_topmem +e801_fail: #ifdef DEBUG_BUILD MSG(int1588_string) #endif @@ -537,16 +570,18 @@ #ifdef DEBUG_BUILD call phword #endif - cmpw $(0x4000-0x400), %ax /* safety check: if it's above 15M ... */ - jbe topmem_ok - movw $(0x4000-0x400), %ax /* ... replace with 15M. */ -topmem_ok: - addw $0x400, %ax /* Adjust for 1M offset */ - shlw $1, %ax /* Convert to sectors */ - subw initrd_sects, %ax /* Compute start of initrd */ - shlw $1, %ax /* Convert to 256-byte blocks */ - andw $0xfff0, %ax /* Round down to 4k block (req'd by kernel) */ - movw %ax, initrd_start /* Store initrd start for later use */ + cmpw %bp, %ax /* safety check: if it's above 15M ... */ + jbe mem_le16meg + movw %bp, %ax /* ... replace with 15M. */ +mem_le16meg: + addw $0x400, %ax /* Add 1M (in 1k blocks) back in. */ + shrw $6, %ax /* Convert to 64k blocks (round down) */ +got_topmem: + movw initrd_sects, %bx + shrw $7, %bx /* Convert to 64k blocks (round down) */ + incw %bx /* Account for the fact that we rounded down. */ + subw %bx, %ax /* Compute start of initrd in 64k blocks */ + movw %ax, initrd_start #ifdef DEBUG_BUILD pushw %ax MSG(initrdstart_string) @@ -563,7 +598,7 @@ */ /* Find top of low memory */ - movw %dx, %ds + popw %ds /* pop 0 */ movw 0x0413, %ax #ifdef DEBUG_BUILD call phword @@ -617,9 +652,10 @@ #ifdef DEBUG_BUILD docseg_string: .string "doc_seg = 0x" setupseg_string: .string " setup_seg = 0x" -int1588_string: .string "\n\rint15/88 returns 0x" +int15e801_string: .string "\n\rint15/e801 returns 0x" +int1588_string: .string "FAILED\n\rint15/88 returns 0x" initrdstart_string: .string " initrd_start = 0x" -lowmemtop_string: .string "00\n\rtop of low mem = 0x" +lowmemtop_string: .string "0000\n\rtop of low mem = 0x" handlerseg_string: .string "k handler seg = 0x" presskey_string: .string "\n\r -- Press any key --\n\r" #endif From dedekind at infradead.org Thu Apr 21 12:50:24 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:48 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.40,1.41 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv21769 Modified Files: JFFS3design.tex Log Message: Update and refine the DCP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.40 retrieving revision 1.41 diff -u -r1.40 -r1.41 --- JFFS3design.tex 20 Apr 2005 16:10:38 -0000 1.40 +++ JFFS3design.tex 21 Apr 2005 16:50:20 -0000 1.41 @@ -336,71 +336,112 @@ \begin{itemize} \item are associated with regular files; \item refer all the valid nodes of a regular file inode; -\item allow not to keep in-core references to all the inode data nodes, -but only references to the inode data checkpoints; \item play the role of JFFS2 fragtrees, i.e., allow to quickly locate positions of data nodes for any given file data range. \end{itemize} Each regular file inode is associated with one or more data -checkpoints. Each DCP corresponds to a fixed inode data range of size $R$. -I.e., if the size of the regular file is $< R$, it has -only one associated DCP node. If the size of the file is $> {R}$ but -$< {2R}$ then there will be two associated DCP and so forth. -Obviously, $R$ value is multiple of \texttt{PAGE\_SIZE}. - -Each DCP node carries the following information: -\begin{itemize} -\item index $I$ -which defines the DCP range (the file range described by DCP is -${\lbrack}I{\cdot}R, I{\cdot}(R + 1)\rbrack$); - -\item version, in order to distinguish valid and obsolete DCP -nodes; - -\item the list of data nodes belonging to the DCP range; the list is -sorted by the data node offsets, i.e., is effectively the fragtree of -the DCP range. -\end{itemize} - -The above mentioned list is essentially an array of objects, containing -the following: +checkpoints. The number of the associated DCP depends on the file size. +Small files have only one correspondent DCP, large files may have many +associated data checkpoints. + +Each DCP corresponds to a fixed data range of file $R$, or +\emph{DCP range}. +The idea behind such a splitting is to facilitate: \begin{itemize} -\item the data node range; -\item the data node physical address of Flash. +\item GC and file change speed optimization - we update only few DCP nodes if +a large file is being changed or one of its nodes is being GCed; +\item memory consumption optimization - we don't need to keep in-core +the DCP ranges; +\item DCP composing optimization - DCP entries are sorted in DCP and +it is much faster to sort short arrays rather then long. \end{itemize} -Thus, larger files have more associated DCP nodes. It stands for reason -that when file is changed or GC moves any node of the file, the -corresponding DCP node becomes obsolete and ought to be rewritten. But -only those data checkpoints are rewritten that correspond to the changed -file range. - -With the help of DCP JFFS3 need only keep in-core an array (or more -precisely, a list of arrays) of DCP node references of files. - -The precise value of $R$ depends on the memory consumption requirements. -The following table illustrates the dependency of the consumed memory and $R$ -for different file sizes (we assume each DCP reference takes 4 bytes). +The data structure corresponding to DCP is: +\begin{verbatim} +struct jffs3_data_checkpoint +{ + uint16_t magic; /* Magic bitmask of DCP node. */ + uint16_t index; /* The DCP index. Gives DCP range offset + if multiplied by the DCP range. */ + uint32_t version; /* Helps to differentiate between valid + and obsolete data checkpoints. */ + uint32_t hdr_crc; /* DCP Header CRC32 checksum. */ + uint32_t crc; /* DCP CRC32 checksum. */ + + /* An array of references to nodes corresponding to the + * DCP. The array is sorted by the node range offset in + * ascending order. */ + struct jffs3_dcp_entry entries[]; +} + +struct jffs3_dcp_entry +{ + uint32_t phys_offs; /* The position of the node on Flash. */ + uint32_t offs; /* Offset of the data range the node refers. */ + uint16_t len; /* The length of the node data range. */ +} +\end{verbatim} + +The value of the DCP range $R$ depends on various aspects: +\begin{description} +\item[Memory consumption.] The larger is $R$, the fewer DCP references +we should keep in-core. For example, for 128~MiB file +we would need to keep 8192 DCP references in RAM if $R$~=~16~KiB and +only 128 DCP references if $R$~=~1~MiB. + +\item[The maximal DCP node size on flash.] DCP node physical sizes shouldn't be +neither too large nor too small. Very large DCP nodes result in slow +DCP updates whereas very small DCP nodes imply larger space overhead and +almost do not decrease DCP update time. It it sane to limit the DCP +node size by one of few NAND pages. Fore a data checkpoint +of size = 1 NAND page the possible maximal number of DCP entries $E$ +(which are 6~bytes in size) is: \begin{center} -\begin{tabular}{lll} -$R$ & File size & RAM required\\ +\begin{tabular}{ll} +NAND page size & Max. entries per DCP($E$) \\ +512 bytes & 80\\ +2048 bytes & 336\\ +4096 bytes & 667\\ +\end{tabular} +\end{center} + +\item[The maximal data node range.] The larger is the maximal data node +range, the fewer DCP entries we need to store for in a DCP for a fixed $R$. +The following are examples of possible configurations, assuming the +physical DCP node size is limited to the size of the NAND page. + +\begin{center} +\begin{tabular}{ll} +Max. data node range ($r$) & Max. DCP range ($R$)\\ +512 bytes per NAND page &\\ +\hline +4 KiB & 320 KiB\\ +8 KiB & 640 KiB\\ +16 KiB & 1280 KiB\\ +2048 bytes per NAND page &\\ +\hline +4 KiB & 1344 KiB\\ +8 KiB & 2688 KiB\\ +16 KiB & 5376 KiB\\ +4096 bytes per NAND page &\\ \hline -512 KiB & 128 MiB & 1 MiB\\ -1 MiB & 128 MiB & 0.5 MiB\\ -4 MiB & 4 GiB & 4 MiB\\ -8 MiB & 4 GiB & 1 MiB\\ +4 KiB & 2668 KiB\\ +8 KiB & 5336 KiB\\ +16 KiB & 10672 KiB\\ \end{tabular} \end{center} -The $R$ value may be determined from the flash size and the desired RAM -consumption limit and may be configurable. +In order to make it possible to write in smaller chunks then $r$, we +ought to have \hbox{$R > r{\cdot}E$}. Since the number of data nodes +corresponding to an inode range is still limited, JFFS3 must start +merging small data nodes when the number of DCP entries reaches the +value $E$ while not the whole range $R$ is filled by data. The possible +way to calculate $R$ is to use formula +\hbox{$R = (r{\cdot}E)/k$}, where $k=2, 3, ...$ +\end{description} -Since most files on the file system are small in size, we may facilitate -faster changing of small files by making $R$ smaller for the beginning -of files and and larger for the rest of files. I.e, 256~KiB for file -ranges before 2~MiB and 8~MiB for ranges after 2~MiB. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Fri Apr 22 13:18:42 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:48 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.41,1.42 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv26465 Modified Files: JFFS3design.tex Log Message: Update DCP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.41 retrieving revision 1.42 diff -u -r1.41 -r1.42 --- JFFS3design.tex 21 Apr 2005 16:50:20 -0000 1.41 +++ JFFS3design.tex 22 Apr 2005 17:18:39 -0000 1.42 @@ -9,6 +9,7 @@ \documentclass[12pt,a4paper,oneside,titlepage]{article} \usepackage{hyperref} \usepackage{html} +\usepackage{longtable} \textwidth=18cm \oddsidemargin=0cm \topmargin=0cm @@ -335,8 +336,8 @@ Data checkpoints (DCP): \begin{itemize} \item are associated with regular files; -\item refer all the valid nodes of a regular file inode; -\item play the role of JFFS2 fragtrees, i.e., allow to quickly locate +\item refer all the valid nodes of the regular file inode; +\item play the role of JFFS2 fragment trees, i.e., allow to quickly locate positions of data nodes for any given file data range. \end{itemize} @@ -345,15 +346,15 @@ Small files have only one correspondent DCP, large files may have many associated data checkpoints. -Each DCP corresponds to a fixed data range of file $R$, or -\emph{DCP range}. +Each DCP corresponds to a fixed data range of the file, and +this range is called \emph{DCP range} or $R_{DCP}$. The idea behind such a splitting is to facilitate: \begin{itemize} -\item GC and file change speed optimization - we update only few DCP nodes if +\item GC and file change speed optimization -- we update only few DCP nodes if a large file is being changed or one of its nodes is being GCed; -\item memory consumption optimization - we don't need to keep in-core +\item memory consumption optimization -- we don't need to keep in-core the DCP ranges; -\item DCP composing optimization - DCP entries are sorted in DCP and +\item DCP composing optimization -- DCP entries are sorted and it is much faster to sort short arrays rather then long. \end{itemize} @@ -383,65 +384,117 @@ } \end{verbatim} -The value of the DCP range $R$ depends on various aspects: +The value of the DCP range $R_{DCP}$ depends on various aspects: \begin{description} -\item[Memory consumption.] The larger is $R$, the fewer DCP references -we should keep in-core. For example, for 128~MiB file -we would need to keep 8192 DCP references in RAM if $R$~=~16~KiB and -only 128 DCP references if $R$~=~1~MiB. - -\item[The maximal DCP node size on flash.] DCP node physical sizes shouldn't be -neither too large nor too small. Very large DCP nodes result in slow -DCP updates whereas very small DCP nodes imply larger space overhead and -almost do not decrease DCP update time. It it sane to limit the DCP -node size by one of few NAND pages. Fore a data checkpoint -of size = 1 NAND page the possible maximal number of DCP entries $E$ -(which are 6~bytes in size) is: - +\item[Memory consumption.] JFFS3 ought to keep in-core DCP references +and it is clear then the larger is $R_{DCP}$ the less is the memory +consumption. To facilitate good scalability $R_{DCP}$ must grow with growing +flash size. Nonetheless, this will make troubles when moving an JFFS3 +image from small flash to large. + +\item[RAM page size.] Obviously $R_{DCP}$ must be multiple of +\texttt{PAGE\_SIZE}, and to facilitate JFFS3 portability we should +make it multiple of the largest 64~KiB \texttt{PAGE\_SIZE} +used in Linux our days. 4 KiB should be treated as the smallest possible +RAM page size. + +\item[DCP node size.] The DCP node physical size shouldn't be +neither too large nor too small. To facilitate faster DCP update on NAND +flashes the maximum DCP node physical size $S_{DCP}$ should be one NAND +page. +One DCP entry takes 6 bytes and the DCP header takes 32 bytes and it is +easy to calculate the maximum number DCP entries $E_{DCP}$ if DCP node +size would be limited to one NAND page: \begin{center} \begin{tabular}{ll} -NAND page size & Max. entries per DCP($E$) \\ +\textbf{NAND page size} & $\bf E_{DCP}$ \\ +\hline 512 bytes & 80\\ 2048 bytes & 336\\ 4096 bytes & 667\\ \end{tabular} \end{center} -\item[The maximal data node range.] The larger is the maximal data node -range, the fewer DCP entries we need to store for in a DCP for a fixed $R$. -The following are examples of possible configurations, assuming the -physical DCP node size is limited to the size of the NAND page. +\item[Largest data node size.] To gain write performance JFFS3 should be +able to write non-pristine nodes, i.e., nodes of size < +\texttt{PAGE\_SIZE}. In order to allow filling the DCP range by +non-pristine nodes the following should be true: +$$R_{DCP} > R_{data}{\cdot}E_{DCP},$$ +where $R_{data}$ is the maximal data node range and $E_{DCP}$ is the +maximal number of DCP entries in DCP. The table below illustrates +the average data node range size depending on $R$ and NAND page size. +512 bytes NAND page: \begin{center} \begin{tabular}{ll} -Max. data node range ($r$) & Max. DCP range ($R$)\\ -512 bytes per NAND page &\\ +$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -4 KiB & 320 KiB\\ -8 KiB & 640 KiB\\ -16 KiB & 1280 KiB\\ -2048 bytes per NAND page &\\ -\hline -4 KiB & 1344 KiB\\ -8 KiB & 2688 KiB\\ -16 KiB & 5376 KiB\\ -4096 bytes per NAND page &\\ +64 KiB & 820 bytes\\ +128 KiB & 1.64 KiB\\ +256 KiB & 3.28 KiB\\[4pt] +\end{tabular} +\end{center} + +2048 bytes NAND page: +\begin{center} +\begin{tabular}{ll} +$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -4 KiB & 2668 KiB\\ -8 KiB & 5336 KiB\\ -16 KiB & 10672 KiB\\ +256 KiB & 780 bytes\\ +512 KiB & 1.56 KiB\\ +1 MiB & 3.12 KiB\\[4pt] \end{tabular} \end{center} -In order to make it possible to write in smaller chunks then $r$, we -ought to have \hbox{$R > r{\cdot}E$}. Since the number of data nodes -corresponding to an inode range is still limited, JFFS3 must start -merging small data nodes when the number of DCP entries reaches the -value $E$ while not the whole range $R$ is filled by data. The possible -way to calculate $R$ is to use formula -\hbox{$R = (r{\cdot}E)/k$}, where $k=2, 3, ...$ +4096 bytes NAND page: +\begin{center} +\begin{tabular}{ll} +$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ +\hline +512 KiB & 790 bytes\\ +1 MiB & 1.57 KiB\\ +2 MiB & 3.14 KiB\\ +\end{tabular} +\end{center} \end{description} +Keeping in-core references to all the DCP nodes of a +file may eat much RAM. Assuming that we keep in-core only 4~byte DCP +flash addresses (which is in fact unreachable) we have: + +\begin{center} +\begin{tabular}{llll} +\textbf{File size} & \textbf{NAND Page size} & +$\bf R_{DCP}$ & \textbf{RAM required}\\ +\hline +64 MiB & 512 bytes & 256 KiB & 1 MiB\\ +1 GiB & 2048 bytes & 1 MiB & 4 MiB\\ +8 GiB & 4096 bytes & 2 MiB & 8 MiB\\ +\end{tabular} +\end{center} + +To relax this JFFS3 uses \emph{second level data checkpoints} (DCP2) referring +the level one data checkpoints of the file. Similarly to DCP, DCP2 +constraints are: +\begin{itemize} +\item DCP2 range $R_{DCP2}$ is multiple of DCP range $R_{DCP}$; +\item the largest DCP2 node physical size $S_{DCP2}$ is limited by the +NAND page size; +\end{itemize} + +With DCP2 the above table looks as: +\begin{center} +\begin{tabular}{llll} +\textbf{File size} & \textbf{NAND Page size} & +$\bf R_{DCP}$ & \textbf{RAM required}\\ +\hline +64 MiB & 512 bytes & 256 KiB & 12 bytes\\ +1 GiB & 2048 bytes & 1 MiB & 12 bytes\\ +8 GiB & 4096 bytes & 2 MiB & 24 bytes\\ +\end{tabular} +\end{center} + +TODO: but the real picture is not so nice... more complications needed... %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % @@ -680,7 +733,7 @@ { list_head *list; node_ref *node; - /* Status bits and accounting dependend on state */ + /* Status bits and accounting dependent on state */ u32 status; } \end{verbatim} @@ -716,7 +769,7 @@ selectable by the user to scale the system according to the requirements. See the paragraph about virtual erase blocks. -Using JFFS2 concatentation of 16 blocks would reduce the RAM +Using JFFS2 concatenation of 16 blocks would reduce the RAM requirement to 32KiB for the largest devices. \subsection{JFFS2 blocks reference} \label{ref_JFFS2_blk_ref} @@ -775,7 +828,7 @@ this depends on several aspects: \begin{itemize} -\item The physcial block size. Nodes must not exceed the physical block size +\item The physical block size. Nodes must not exceed the physical block size to allow the configurable selection of block concatenation in virtual blocks. \item Nodes can not be written over physical block boundaries. From dedekind at infradead.org Sat Apr 23 08:33:52 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:48 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.42,1.43 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv29649 Modified Files: JFFS3design.tex Log Message: Update DCP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.42 retrieving revision 1.43 diff -u -r1.42 -r1.43 --- JFFS3design.tex 22 Apr 2005 17:18:39 -0000 1.42 +++ JFFS3design.tex 23 Apr 2005 12:33:49 -0000 1.43 @@ -352,7 +352,7 @@ \begin{itemize} \item GC and file change speed optimization -- we update only few DCP nodes if a large file is being changed or one of its nodes is being GCed; -\item memory consumption optimization -- we don't need to keep in-core +\item memory consumption optimization -- JFFS3 doesn't need to keep in-core the DCP ranges; \item DCP composing optimization -- DCP entries are sorted and it is much faster to sort short arrays rather then long. @@ -374,14 +374,14 @@ * DCP. The array is sorted by the node range offset in * ascending order. */ struct jffs3_dcp_entry entries[]; -} +} __attribute__((packed)); struct jffs3_dcp_entry { uint32_t phys_offs; /* The position of the node on Flash. */ uint32_t offs; /* Offset of the data range the node refers. */ uint16_t len; /* The length of the node data range. */ -} +} __attribute__((packed)); \end{verbatim} The value of the DCP range $R_{DCP}$ depends on various aspects: @@ -392,8 +392,12 @@ flash size. Nonetheless, this will make troubles when moving an JFFS3 image from small flash to large. +From the other hand, very large $R_{DCP}$ implies large physical +DCP nodes size and slower DCP updates. + \item[RAM page size.] Obviously $R_{DCP}$ must be multiple of -\texttt{PAGE\_SIZE}, and to facilitate JFFS3 portability we should +\texttt{PAGE\_SIZE}, and to facilitate JFFS3 portability in the case of +removable devices, we should make it multiple of the largest 64~KiB \texttt{PAGE\_SIZE} used in Linux our days. 4 KiB should be treated as the smallest possible RAM page size. @@ -402,25 +406,24 @@ neither too large nor too small. To facilitate faster DCP update on NAND flashes the maximum DCP node physical size $S_{DCP}$ should be one NAND page. -One DCP entry takes 6 bytes and the DCP header takes 32 bytes and it is -easy to calculate the maximum number DCP entries $E_{DCP}$ if DCP node -size would be limited to one NAND page: +One DCP entry takes 10 bytes, the DCP node header takes 16 bytes and it is +easy to calculate the maximum number DCP entries $E_{DCP}$: \begin{center} \begin{tabular}{ll} \textbf{NAND page size} & $\bf E_{DCP}$ \\ \hline -512 bytes & 80\\ -2048 bytes & 336\\ -4096 bytes & 667\\ +512 bytes & 49\\ +2048 bytes & 203\\ +4096 bytes & 408\\ \end{tabular} \end{center} -\item[Largest data node size.] To gain write performance JFFS3 should be +\item[Largest data node size.] To gain the write performance JFFS3 should be able to write non-pristine nodes, i.e., nodes of size < \texttt{PAGE\_SIZE}. In order to allow filling the DCP range by non-pristine nodes the following should be true: -$$R_{DCP} > R_{data}{\cdot}E_{DCP},$$ -where $R_{data}$ is the maximal data node range and $E_{DCP}$ is the +$$R_{DCP} > \tt{PAGE_SIZE}{\cdot}E_{DCP},$$, +where $E_{DCP}$ is the maximal number of DCP entries in DCP. The table below illustrates the average data node range size depending on $R$ and NAND page size. @@ -429,9 +432,8 @@ \begin{tabular}{ll} $\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -64 KiB & 820 bytes\\ -128 KiB & 1.64 KiB\\ -256 KiB & 3.28 KiB\\[4pt] +64 KiB & 1.3 KiB\\ +128 KiB & 2.7 KiB\\ \end{tabular} \end{center} @@ -440,9 +442,9 @@ \begin{tabular}{ll} $\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -256 KiB & 780 bytes\\ -512 KiB & 1.56 KiB\\ -1 MiB & 3.12 KiB\\[4pt] +128 KiB & 645 bytes\\ +256 KiB & 1.3 KiB\\ +512 KiB & 2.6 KiB\\ \end{tabular} \end{center} @@ -451,31 +453,45 @@ \begin{tabular}{ll} $\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -512 KiB & 790 bytes\\ -1 MiB & 1.57 KiB\\ -2 MiB & 3.14 KiB\\ +256 KiB & 642 bytes\\ +512 KiB & 1.3 KiB\\ +1 MiB & 2.6 KiB\\ \end{tabular} \end{center} + +The above tables assume each DCP entry may at most refer 4~KiB of data. +Thus we guarantee that DCP node won't be larger then one NAND page. + +For NOR and other non-paged flashes JFFS3 just assumes some that +the flash page size is 512 bytes or larger, depending on the +flash size. \end{description} -Keeping in-core references to all the DCP nodes of a -file may eat much RAM. Assuming that we keep in-core only 4~byte DCP -flash addresses (which is in fact unreachable) we have: +If we keep in RAM references of all the DCP nodes of a file, we may +waste much RAM. Assuming that each reference takes 4~bytes +(which is in fact unreachably few) we have: \begin{center} -\begin{tabular}{llll} +\begin{tabular}{lllll} \textbf{File size} & \textbf{NAND Page size} & -$\bf R_{DCP}$ & \textbf{RAM required}\\ +$\bf R_{DCP}$ & \textbf{RAM required} & +\textbf{Flash overhead}\\ \hline -64 MiB & 512 bytes & 256 KiB & 1 MiB\\ -1 GiB & 2048 bytes & 1 MiB & 4 MiB\\ -8 GiB & 4096 bytes & 2 MiB & 8 MiB\\ +64 MiB & 512 bytes & 128 KiB & 2 MiB & 256 KiB (0.4\%)\\ +1 GiB & 2048 bytes & 512 KiB & 8 MiB & 4 MiB (0.4\%)\\ +8 GiB & 4096 bytes & 1 MiB & 32 MiB & 16 MiB (0.4\%)\\ \end{tabular} \end{center} -To relax this JFFS3 uses \emph{second level data checkpoints} (DCP2) referring -the level one data checkpoints of the file. Similarly to DCP, DCP2 -constraints are: +The above table also shows the amount of Flash space required to store +DCP nodes (assuming each DCP entry takes the whole Flash page). + + +To relax RAM usage JFFS3 makes use of \emph{second level data +checkpoints} (DCP2) referring +the level one data checkpoints of the file similerly to how DCP refers +data nodes. The following requirements are to be met for DCP2: + \begin{itemize} \item DCP2 range $R_{DCP2}$ is multiple of DCP range $R_{DCP}$; \item the largest DCP2 node physical size $S_{DCP2}$ is limited by the @@ -488,13 +504,12 @@ \textbf{File size} & \textbf{NAND Page size} & $\bf R_{DCP}$ & \textbf{RAM required}\\ \hline -64 MiB & 512 bytes & 256 KiB & 12 bytes\\ -1 GiB & 2048 bytes & 1 MiB & 12 bytes\\ -8 GiB & 4096 bytes & 2 MiB & 24 bytes\\ +64 MiB & 512 bytes & 256 KiB & 44 bytes\\ +1 GiB & 2048 bytes & 1 MiB & 44 bytes\\ +8 GiB & 4096 bytes & 2 MiB & 84 bytes\\ \end{tabular} \end{center} -TODO: but the real picture is not so nice... more complications needed... %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Sat Apr 23 10:48:55 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:48 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.43,1.44 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv29976 Modified Files: JFFS3design.tex Log Message: Update DCP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.43 retrieving revision 1.44 diff -u -r1.43 -r1.44 --- JFFS3design.tex 23 Apr 2005 12:33:49 -0000 1.43 +++ JFFS3design.tex 23 Apr 2005 14:48:51 -0000 1.44 @@ -360,12 +360,12 @@ The data structure corresponding to DCP is: \begin{verbatim} -struct jffs3_data_checkpoint +struct jffs3_raw_data_checkpoint { uint16_t magic; /* Magic bitmask of DCP node. */ uint16_t index; /* The DCP index. Gives DCP range offset if multiplied by the DCP range. */ - uint32_t version; /* Helps to differentiate between valid + uint64_t version; /* Helps to differentiate between valid and obsolete data checkpoints. */ uint32_t hdr_crc; /* DCP Header CRC32 checksum. */ uint32_t crc; /* DCP CRC32 checksum. */ @@ -376,7 +376,7 @@ struct jffs3_dcp_entry entries[]; } __attribute__((packed)); -struct jffs3_dcp_entry +struct jffs3_raw_dcp_entry { uint32_t phys_offs; /* The position of the node on Flash. */ uint32_t offs; /* Offset of the data range the node refers. */ @@ -406,15 +406,15 @@ neither too large nor too small. To facilitate faster DCP update on NAND flashes the maximum DCP node physical size $S_{DCP}$ should be one NAND page. -One DCP entry takes 10 bytes, the DCP node header takes 16 bytes and it is +One DCP entry takes 10 bytes, the DCP node header takes 20 bytes and it is easy to calculate the maximum number DCP entries $E_{DCP}$: \begin{center} \begin{tabular}{ll} \textbf{NAND page size} & $\bf E_{DCP}$ \\ \hline 512 bytes & 49\\ -2048 bytes & 203\\ -4096 bytes & 408\\ +2048 bytes & 202\\ +4096 bytes & 407\\ \end{tabular} \end{center} @@ -422,7 +422,7 @@ able to write non-pristine nodes, i.e., nodes of size < \texttt{PAGE\_SIZE}. In order to allow filling the DCP range by non-pristine nodes the following should be true: -$$R_{DCP} > \tt{PAGE_SIZE}{\cdot}E_{DCP},$$, +$$R_{DCP} > \mathtt{PAGE\_SIZE}{\cdot}E_{DCP},$$ where $E_{DCP}$ is the maximal number of DCP entries in DCP. The table below illustrates the average data node range size depending on $R$ and NAND page size. @@ -442,7 +442,7 @@ \begin{tabular}{ll} $\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline -128 KiB & 645 bytes\\ +128 KiB & 650 bytes\\ 256 KiB & 1.3 KiB\\ 512 KiB & 2.6 KiB\\ \end{tabular} @@ -462,8 +462,8 @@ The above tables assume each DCP entry may at most refer 4~KiB of data. Thus we guarantee that DCP node won't be larger then one NAND page. -For NOR and other non-paged flashes JFFS3 just assumes some that -the flash page size is 512 bytes or larger, depending on the +For NOR and other non-paged flashes JFFS3 just assumes that +the flash page size is virtually 512 bytes or larger, depending on the flash size. \end{description} @@ -486,10 +486,9 @@ The above table also shows the amount of Flash space required to store DCP nodes (assuming each DCP entry takes the whole Flash page). - To relax RAM usage JFFS3 makes use of \emph{second level data checkpoints} (DCP2) referring -the level one data checkpoints of the file similerly to how DCP refers +the level one data checkpoints of the file similarly to how DCP refers data nodes. The following requirements are to be met for DCP2: \begin{itemize} @@ -500,16 +499,21 @@ With DCP2 the above table looks as: \begin{center} -\begin{tabular}{llll} +\begin{tabular}{lllll} \textbf{File size} & \textbf{NAND Page size} & -$\bf R_{DCP}$ & \textbf{RAM required}\\ +$\bf R_{DCP}$ & $\bf R_{DCP2}$ +& \textbf{RAM required}\\ \hline -64 MiB & 512 bytes & 256 KiB & 44 bytes\\ -1 GiB & 2048 bytes & 1 MiB & 44 bytes\\ -8 GiB & 4096 bytes & 2 MiB & 84 bytes\\ +64 MiB & 512 bytes & 256 KiB & 8 MiB & 32 bytes\\ +1 GiB & 2048 bytes & 1 MiB & 128 MiB & 32 bytes\\ +8 GiB & 4096 bytes & 2 MiB & 512 MiB & 64 bytes\\ \end{tabular} \end{center} +Besides $R_{DCP2}$ JFFS3 defines one more constant $R_{DCP2}^{min}$ +denoting the minimal size of file which can have an +associated DCP2 node. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % From dedekind at infradead.org Sat Apr 23 11:07:13 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:49 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.44,1.45 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv30041 Modified Files: JFFS3design.tex Log Message: Remove outdated info. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.44 retrieving revision 1.45 diff -u -r1.44 -r1.45 --- JFFS3design.tex 23 Apr 2005 14:48:51 -0000 1.44 +++ JFFS3design.tex 23 Apr 2005 15:07:09 -0000 1.45 @@ -517,147 +517,6 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % -% CHECKSUM -% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Checksum} -Any data (including actual file content and internal JFFS3 data) stored within -JFFS3 file-system is protected by checksum. Nothing is stored on the flash media -without checksum. - -Checksums are destined to detect errors caused by wearing, BB appearance -or external factors (radiation, abnormal temperatures and the -like). Errors fall into the following categories. - -\begin{itemize} -\item \emph{Permanent} error is an error which stays after the corresponding block has -been erased. - -\item \emph{Occasional} error is an error which becomes fixed after the corresponding -block has been erased. -\end{itemize} - -Different flash technologies imply different reliability. The following -is a brief overview of Flash technologies in this perspective. - -\begin{itemize} -\item \emph{NOR} -flash is the most trustworthy type of flashes which doesn't admit of permanent -errors. Any permanent error is critical and means that the chip is not workable anymore. -Nevertheless, some kinds of NOR flash are not avoid of very rare bit flipping -events and the use of checksum still might make sense here. - -\item \emph{DataFlash} is based on NOR technology and inherits its properties. -[\ref{ref_AtmelDataFlash}]. - -\item \emph{ECC NOR} may be regarded as not very robust NOR flash plus extra ECC -protection. - -\item \emph{NAND} flash is not so reliable as NOR and it admits of permanent errors, let -alone occasional ones which are also possible. The NAND technology itself -implies bad blocks may dynamically appear during its utilization, albeit this is -rather rare event [\ref{ref_NANDMTD}, \ref{ref_ToshibaNAND}]. - -\item \emph{OneNAND} is based on NAND technology and has similar properties -[\ref{ref_SamsungOneNAND}]. - -\item \emph{AND} (\emph{AG-AND} and \emph{SuperAND}), -like NAND technology, implies bad blocks either factory marked -or dynamically found [\ref{ref_RenesasAND}]. -\end{itemize} - -% -% Checksum modes -% -\subsection{Checksum modes} -Because of a significant disparity in the reliability between different types of -flashes JFFS3 might operate in either of the following modes. - -\begin{enumerate} -\item \emph{Strict mode} -- checksums are always verified i.e. anything JFFS2 -reads from the flash is validated by checksum. This is safest operation mode and -should probably always be enabled by default at least with NAND/AND flashes. - -\item \emph{Relaxed mode} -- data checksums are not ever being checked, i.e -JFFS3 still checks node headers and other internal data but doesn't check actual -file's content. This is dangerous working mode and may probably only be enabled -by default when JFFS3 works on top of NOR flash. -\end{enumerate} - -Both of the above mentioned operation modes imply that JFFS3 calculates and -saves checksums when it is performing write operations. - -If the underlying flash makes use of ECC (e.g., NAND), and JFFS3 has been -reported ECC error, JFFS3 checks the corresponding data that had been read. - -The checksum mode might be set on the per file basis with -help of the corresponding extended attribute. In this case the extended -attribute overrides the default mode. This is very handy mechanism -whereby users might, for example, disable the checking of multimedia -file's content. - -% -% Checksum and unclean reboots -% -\subsection{Checksum and unclean reboots} -Unlike JFFS2, JFFS3 does not utilize checksum to detect unclean reboots, albeit -might do this. The primary aim of checksum is to prevent problems -related to physical media corruptions. Unclean reboots are substantially -detected by means of summary nodes. This property makes it possible to distinct -between checksum errors caused by media problems and by unclean reboots. - -% -% Checksum errors -% -\subsection{Checksum errors} -If a checksum error has been found and if it has not been caused by -an unclean reboot, JFFS3 behaves as follows. - -\begin{enumerate} -\item Reports the fact of the checksum error. - -\item Takes an attempt to recover the erroneous node. -Ignores the erroneous node (possibly, marking it obsolete) irrespective -of the recovery procedure result (success or failure). - -\item Locks the inode to whom the erroneous node belongs. Namely, -rejects any further access to the inode. This might be implementing by -means of setting the relating extended attribute. JFFS3 -must not reject xattr operation whereby user might clean the -attribute that locks the inode. - -\item Returns an error if the operation was being done on behalf of an -user process (not Garbage Collector). Otherwise, let the GC to carry on. -\end{enumerate} - -% -% Checksum error recovery -% -\subsubsection{Checksum error recovery} -The recovery procedure -has not been designed so far and probably we may postpone this work as -unimportant although it should be kept in mind during JFFS3 -implementation. - -Conceivably, JFFS3 might try to find an older obsolete node -containing the corrupted information as well as to reconstruct the -corrupted data using the corresponding in-memory data if it is present. -The recovery is especially important in case of corrupted direntry node -as if we just ignore it we might loose the whole file or directory with -children. - -% -% Miscellaneous -% -\subsection{Miscellaneous} -Like JFFS2, JFFS3 makes use of CRC32 checksum algorithm with the start seed -0xFFFFFFFF. - -If zlib library is used to compress node's data it is worth asking zlib not to -generate redundant adler32 checksum for better performance. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% % VIRTUAL BLOCK HANDLING % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -835,6 +694,148 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % +% CHECKSUM +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\section{Checksum} +Any data (including actual file content and internal JFFS3 data) stored within +JFFS3 file-system is protected by checksum. Nothing is stored on the flash media +without checksum. + +Checksums are destined to detect errors caused by wearing, BB appearance +or external factors (radiation, abnormal temperatures and the +like). Errors fall into the following categories. + +\begin{itemize} +\item \emph{Permanent} error is an error which stays after the corresponding block has +been erased. + +\item \emph{Occasional} error is an error which becomes fixed after the corresponding +block has been erased. +\end{itemize} + +Different flash technologies imply different reliability. The following +is a brief overview of Flash technologies in this perspective. + +\begin{itemize} +\item \emph{NOR} +flash is the most trustworthy type of flashes which doesn't admit of permanent +errors. Any permanent error is critical and means that the chip is not workable anymore. +Nevertheless, some kinds of NOR flash are not avoid of very rare bit flipping +events and the use of checksum still might make sense here. + +\item \emph{DataFlash} is based on NOR technology and inherits its properties. +[\ref{ref_AtmelDataFlash}]. + +\item \emph{ECC NOR} may be regarded as not very robust NOR flash plus extra ECC +protection. + +\item \emph{NAND} flash is not so reliable as NOR and it admits of permanent errors, let +alone occasional ones which are also possible. The NAND technology itself +implies bad blocks may dynamically appear during its utilization, albeit this is +rather rare event [\ref{ref_NANDMTD}, \ref{ref_ToshibaNAND}]. + +\item \emph{OneNAND} is based on NAND technology and has similar properties +[\ref{ref_SamsungOneNAND}]. + +\item \emph{AND} (\emph{AG-AND} and \emph{SuperAND}), +like NAND technology, implies bad blocks either factory marked +or dynamically found [\ref{ref_RenesasAND}]. +\end{itemize} + +% +% Checksum modes +% +\subsection{Checksum modes} +Because of a significant disparity in the reliability between different types of +flashes JFFS3 might operate in either of the following modes. + +\begin{enumerate} +\item \emph{Strict mode} -- checksums are always verified i.e. anything JFFS2 +reads from the flash is validated by checksum. This is safest operation mode and +should probably always be enabled by default at least with NAND/AND flashes. + +\item \emph{Relaxed mode} -- data checksums are not ever being checked, i.e +JFFS3 still checks node headers and other internal data but doesn't check actual +file's content. This is dangerous working mode and may probably only be enabled +by default when JFFS3 works on top of NOR flash. +\end{enumerate} + +Both of the above mentioned operation modes imply that JFFS3 calculates and +saves checksums when it is performing write operations. + +If the underlying flash makes use of ECC (e.g., NAND), and JFFS3 has been +reported ECC error, JFFS3 checks the corresponding data that had been read. + +The checksum mode might be set on the per file basis with +help of the corresponding extended attribute. In this case the extended +attribute overrides the default mode. This is very handy mechanism +whereby users might, for example, disable the checking of multimedia +file's content. + +% +% Checksum and unclean reboots +% +\subsection{Checksum and unclean reboots} +Unlike JFFS2, JFFS3 does not utilize checksum to detect unclean reboots, albeit +might do this. The primary aim of checksum is to prevent problems +related to physical media corruptions. Unclean reboots are substantially +detected by means of summary nodes. This property makes it possible to distinct +between checksum errors caused by media problems and by unclean reboots. + +% +% Checksum errors +% +\subsection{Checksum errors} +If a checksum error has been found and if it has not been caused by +an unclean reboot, JFFS3 behaves as follows. + +\begin{enumerate} +\item Reports the fact of the checksum error. + +\item Takes an attempt to recover the erroneous node. +Ignores the erroneous node (possibly, marking it obsolete) irrespective +of the recovery procedure result (success or failure). + +\item Locks the inode to whom the erroneous node belongs. Namely, +rejects any further access to the inode. This might be implementing by +means of setting the relating extended attribute. JFFS3 +must not reject xattr operation whereby user might clean the +attribute that locks the inode. + +\item Returns an error if the operation was being done on behalf of an +user process (not Garbage Collector). Otherwise, let the GC to carry on. +\end{enumerate} + +% +% Checksum error recovery +% +\subsubsection{Checksum error recovery} +The recovery procedure +has not been designed so far and probably we may postpone this work as +unimportant although it should be kept in mind during JFFS3 +implementation. + +Conceivably, JFFS3 might try to find an older obsolete node +containing the corrupted information as well as to reconstruct the +corrupted data using the corresponding in-memory data if it is present. +The recovery is especially important in case of corrupted direntry node +as if we just ignore it we might loose the whole file or directory with +children. + +% +% Miscellaneous +% +\subsection{Miscellaneous} +Like JFFS2, JFFS3 makes use of CRC32 checksum algorithm with the start seed +0xFFFFFFFF. + +If zlib library is used to compress node's data it is worth asking zlib not to +generate redundant adler32 checksum for better performance. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% % MAXIMAL SIZE OF THE INODE NODE DATA % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -921,161 +922,6 @@ drawback, see (\ref{ref_JFFS2_and_mem}) for more information about this. \end{itemize} -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% -% INODE CHECKPOINTS -% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Inode Checkpoints} - -\subsection{JFFS2 and memory consumption problem} \label{ref_JFFS2_and_mem} -TODO. - -Take a glimpse at\\ -\url{http://lists.infradead.org/pipermail/linux-mtd/2005-January/011671.html} -for the description of the problem for now. - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% -% DATA STRUCTURES -% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Data Structures} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% -% ALGORITHMS -% -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Algorithms} -This section describes the main JFFS3 algorithms in pseudo-code. The -description doesn't contain all the details but refers most fundamental -steps. - -\subsection{Inode build} -The algorithm is initiated by the \texttt{iget()} VFS call. From the user's -perspective it happens e.g. on \texttt{stat()} and \texttt{open()} calls. - -\begin{verbatim} -For each node_ref -{ - read the corresponding node from the flash; - If the inode is the directory inode - { - insert the dirent node to dentree. - } - Else if the inode is the regular file inode - { - insert the data node to fragtree. - } - Else if the inode is the symlink inode - { - Read the symlink inode node and cache the symlink's target; - } - Else read the metadata for another types of the inode as well; -} - -If the inode is the directory inode or the regular file inode -{ - For each icp_ref - { - read the corresponding ICP from the flash; - For each node in this ICP - { - If the inode is the directory inode - { - insert the dirent node to dentree. - } - Else if the inode is the regular file inode - { - insert the data node to fragtree. - } - } - } -} -\end{verbatim} - -Notes: -\begin{itemize} -\item Since the nodes which aren't referred by any ICP are younger (have -the higher version) then those which refereed, we start building fragtree -from them. In this case, we will have fewer RB-tree balancing -operations. - -\item ICP entries are sorted by versions in ICPs and they must be -processed in the descended version order. -\end{itemize} - -\subsection{Garbage Collection} -\begin{verbatim} -Choose the vblock to GC (gcblock); -Read the gcblock's summary; - -For each node in the gcblock -{ - If node is valid - { - move the node; - If the node is described by an ICP - { - Obsolete the corresponding ICP entry in the ICP; - } - } -} -\end{verbatim} - -Notes: -\begin{itemize} -\item See [\ref{ref_Node_Valid}] for the algorithm of checking whether a -node is valid; - -\item See [\ref{ref_Move_Node}] for the algorithm of a node moving; - -\item See [\ref{ref_Obsolete_Entry}] for the algorithm of how to -obsolete an ICP entry in the ICP. -\end{itemize} - -\subsubsection{Check if node is valid} \label{ref_Node_Valid} -Each vblock has an associated list of its obsolete nodes. To check if -a vblock's node is valid the list should be scanned. - -\subsubsection{Move a node} \label{ref_Move_Node} -TODO - -\subsection{Obsolete an ICP entry} \label{ref_Obsolete_Entry} -An ICP is associated with the icp\_ref. One inode may have more then on -ICP. - -An ICP covers some version range and includes ICP entries referring -\emph{all} the \emph{valid} nodes within that range. - -An icp\_ref contains a counter of valid nodes that it refers. When one -of its nodes becomes obsolete, the counter is decremented. When the -number of valid nodes referred by the ICP becomes too small, the ICP is -treated as obsolete and is removed from the inode's icp\_ref list. - -\begin{verbatim} -Decrement the icp_ref's valid nodes count; - -If the count becomes too small -{ - For each valid node in the icp - { - Allocate the node_ref for it; - Insert this node_ref to the node_refs list; - } - - Remove the icp_ref from the icp_ref list; - Add the icp_ref's node_ref to the list of obsolete nodes; -} -\end{verbatim} - -\subsection{Flash scan} - -\subsection{FS build} - -\subsection{Memory shrink} -TODO %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % @@ -1086,13 +932,16 @@ \begin{enumerate} \item \emph{BB} - Bad Block \item \emph{CRC} - Cyclic Redundancy Check -\item \emph{ICP} - Inode Check-Point +\item \emph{CP} - Check-Point +\item \emph{DCP} - Data Check-Point +\item \emph{DCP2} - Data Check-Point level 2 \item \emph{JFFS2} - Journalling Flash File System version 2 \item \emph{JFFS3} - Journalling Flash File System version 3 \item \emph{MTD} - Memory Technology Devices \item \emph{VFS} - Virtual File System \end{enumerate} + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % DEFINITIONS From dedekind at infradead.org Sat Apr 23 13:00:08 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:49 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.45,1.46 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv30293 Modified Files: JFFS3design.tex Log Message: Update DCP chapter. Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.45 retrieving revision 1.46 diff -u -r1.45 -r1.46 --- JFFS3design.tex 23 Apr 2005 15:07:09 -0000 1.45 +++ JFFS3design.tex 23 Apr 2005 17:00:04 -0000 1.46 @@ -50,7 +50,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % -% JFFS2 analysis +% JFFS2 ANALYSIS % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{JFFS2 analysis} @@ -325,58 +325,58 @@ types of checkpoints are defined: \begin{itemize} -\item data checkpoints which are associated with regular files; -\item directory checkpoints which are associated with directories. +\item file checkpoints which are associated with regular file inodes; +\item directory checkpoints which are associated with directory inodes. \end{itemize} % % Data checkpoints % -\subsection{Data checkpoints} -Data checkpoints (DCP): +\subsection{File checkpoints} +File checkpoints (FCP): \begin{itemize} -\item are associated with regular files; -\item refer all the valid nodes of the regular file inode; +\item are associated with regular file inodes; +\item refer all the valid data nodes of the file; \item play the role of JFFS2 fragment trees, i.e., allow to quickly locate positions of data nodes for any given file data range. \end{itemize} -Each regular file inode is associated with one or more data -checkpoints. The number of the associated DCP depends on the file size. -Small files have only one correspondent DCP, large files may have many -associated data checkpoints. +Each regular file inode is associated with one or more +checkpoints. The number of the associated FCP depends on the file size. +Small files have only one correspondent FCP, large files may have many +associated checkpoints. -Each DCP corresponds to a fixed data range of the file, and -this range is called \emph{DCP range} or $R_{DCP}$. +Each FCP corresponds to a fixed data range of the file, and +this range is called \emph{FCP range} or $R_{FCP}$. The idea behind such a splitting is to facilitate: \begin{itemize} -\item GC and file change speed optimization -- we update only few DCP nodes if -a large file is being changed or one of its nodes is being GCed; +\item fast GC and file change operations -- we update only one or few FCP nodes if +a large file is changed or one of its nodes are Garbage Collected; \item memory consumption optimization -- JFFS3 doesn't need to keep in-core -the DCP ranges; -\item DCP composing optimization -- DCP entries are sorted and -it is much faster to sort short arrays rather then long. +the FCP ranges; +\item FCP creation optimization -- FCP entries are sorted in FCP and +it is faster to sort short arrays rather then long. \end{itemize} -The data structure corresponding to DCP is: +The data structure corresponding to FCP is: \begin{verbatim} -struct jffs3_raw_data_checkpoint +struct jffs3_raw_file_checkpoint { - uint16_t magic; /* Magic bitmask of DCP node. */ - uint16_t index; /* The DCP index. Gives DCP range offset - if multiplied by the DCP range. */ + uint16_t magic; /* Magic bitmask of FCP node. */ + uint16_t index; /* The FCP index. Gives FCP range offset + if multiplied by the FCP range. */ uint64_t version; /* Helps to differentiate between valid and obsolete data checkpoints. */ - uint32_t hdr_crc; /* DCP Header CRC32 checksum. */ - uint32_t crc; /* DCP CRC32 checksum. */ + uint32_t hdr_crc; /* FCP Header CRC32 checksum. */ + uint32_t crc; /* FCP CRC32 checksum. */ /* An array of references to nodes corresponding to the - * DCP. The array is sorted by the node range offset in + * FCP. The array is sorted by the node range offset in * ascending order. */ - struct jffs3_dcp_entry entries[]; + struct jffs3_fcp_entry entries[]; } __attribute__((packed)); -struct jffs3_raw_dcp_entry +struct jffs3_raw_fcp_entry { uint32_t phys_offs; /* The position of the node on Flash. */ uint32_t offs; /* Offset of the data range the node refers. */ @@ -384,33 +384,34 @@ } __attribute__((packed)); \end{verbatim} -The value of the DCP range $R_{DCP}$ depends on various aspects: +The value of the FCP range $R_{FCP}$ depends on various aspects: \begin{description} -\item[Memory consumption.] JFFS3 ought to keep in-core DCP references -and it is clear then the larger is $R_{DCP}$ the less is the memory -consumption. To facilitate good scalability $R_{DCP}$ must grow with growing +\item[Memory consumption.] JFFS3 ought to keep in-core FCP references +and it is clear then the larger is $R_{FCP}$ the less is memory +consumption. To facilitate good scalability $R_{FCP}$ must grow with growing flash size. Nonetheless, this will make troubles when moving an JFFS3 image from small flash to large. -From the other hand, very large $R_{DCP}$ implies large physical -DCP nodes size and slower DCP updates. +From the other hand, very large $R_{FCP}$ implies large physical +FCP nodes size and slower FCP updates. -\item[RAM page size.] Obviously $R_{DCP}$ must be multiple of +\item[RAM page size.] Obviously $R_{FCP}$ must be multiple of \texttt{PAGE\_SIZE}, and to facilitate JFFS3 portability in the case of removable devices, we should -make it multiple of the largest 64~KiB \texttt{PAGE\_SIZE} -used in Linux our days. 4 KiB should be treated as the smallest possible +make it multiple of the largest \texttt{PAGE\_SIZE} +used in Linux our days (64 KiB). 4 KiB should be regarded as the smallest possible RAM page size. -\item[DCP node size.] The DCP node physical size shouldn't be -neither too large nor too small. To facilitate faster DCP update on NAND -flashes the maximum DCP node physical size $S_{DCP}$ should be one NAND -page. -One DCP entry takes 10 bytes, the DCP node header takes 20 bytes and it is -easy to calculate the maximum number DCP entries $E_{DCP}$: +\item[FCP node size.] FCP node physical size shouldn't be +neither too large nor too small. To facilitate faster FCP update on NAND +flashes the maximum FCP node physical size ($S_{FCP}$) should be one NAND +page (or less). + +Since one FCP entry takes 10 bytes, the FCP node header takes 20 bytes +the maximum number of FCP entries $E_{FCP}$ will be: \begin{center} \begin{tabular}{ll} -\textbf{NAND page size} & $\bf E_{DCP}$ \\ +\textbf{NAND page size} & $\bf E_{FCP}$ \\ \hline 512 bytes & 49\\ 2048 bytes & 202\\ @@ -419,18 +420,18 @@ \end{center} \item[Largest data node size.] To gain the write performance JFFS3 should be -able to write non-pristine nodes, i.e., nodes of size < -\texttt{PAGE\_SIZE}. In order to allow filling the DCP range by +able to write non-pristine nodes, i.e., nodes of +\mbox{size < \texttt{PAGE\_SIZE}}. In order to allow filling the FCP range by non-pristine nodes the following should be true: -$$R_{DCP} > \mathtt{PAGE\_SIZE}{\cdot}E_{DCP},$$ -where $E_{DCP}$ is the -maximal number of DCP entries in DCP. The table below illustrates +$$R_{FCP} > \mathtt{PAGE\_SIZE}{\cdot}E_{FCP},$$ +where $E_{FCP}$ is the +maximal number of FCP entries in FCP. The table below illustrates the average data node range size depending on $R$ and NAND page size. 512 bytes NAND page: \begin{center} \begin{tabular}{ll} -$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ +$\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline 64 KiB & 1.3 KiB\\ 128 KiB & 2.7 KiB\\ @@ -440,7 +441,7 @@ 2048 bytes NAND page: \begin{center} \begin{tabular}{ll} -$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ +$\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline 128 KiB & 650 bytes\\ 256 KiB & 1.3 KiB\\ @@ -451,7 +452,7 @@ 4096 bytes NAND page: \begin{center} \begin{tabular}{ll} -$\bf R_{DCP}$ & \textbf{Average $\bf R_{data}$}\\ +$\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ \hline 256 KiB & 642 bytes\\ 512 KiB & 1.3 KiB\\ @@ -459,22 +460,20 @@ \end{tabular} \end{center} -The above tables assume each DCP entry may at most refer 4~KiB of data. -Thus we guarantee that DCP node won't be larger then one NAND page. +The above tables assumei that each FCP entry may at most refer 4~KiB of data. For NOR and other non-paged flashes JFFS3 just assumes that the flash page size is virtually 512 bytes or larger, depending on the flash size. \end{description} -If we keep in RAM references of all the DCP nodes of a file, we may -waste much RAM. Assuming that each reference takes 4~bytes +If we keep in-core references of all the FCP nodes of a file, we may +waste a lot ot RAM. Even assuming that each reference takes 4~bytes (which is in fact unreachably few) we have: - \begin{center} \begin{tabular}{lllll} \textbf{File size} & \textbf{NAND Page size} & -$\bf R_{DCP}$ & \textbf{RAM required} & +$\bf R_{FCP}$ & \textbf{RAM required} & \textbf{Flash overhead}\\ \hline 64 MiB & 512 bytes & 128 KiB & 2 MiB & 256 KiB (0.4\%)\\ @@ -484,24 +483,24 @@ \end{center} The above table also shows the amount of Flash space required to store -DCP nodes (assuming each DCP entry takes the whole Flash page). +FCP nodes (assuming each FCP entry takes the whole Flash page). -To relax RAM usage JFFS3 makes use of \emph{second level data -checkpoints} (DCP2) referring -the level one data checkpoints of the file similarly to how DCP refers -data nodes. The following requirements are to be met for DCP2: +To relax RAM usage JFFS3 makes use of \emph{second level file +checkpoints} (FCP2) referring +the level one file checkpoints of the file similarly to how FCP refers +data nodes. The following requirements are to be met for FCP2: \begin{itemize} -\item DCP2 range $R_{DCP2}$ is multiple of DCP range $R_{DCP}$; -\item the largest DCP2 node physical size $S_{DCP2}$ is limited by the +\item FCP2 range $R_{FCP2}$ is multiple of FCP range $R_{FCP}$; +\item the largest FCP2 node physical size $S_{FCP2}$ is limited by the NAND page size; \end{itemize} -With DCP2 the above table looks as: +With FCP2 the above table looks as: \begin{center} \begin{tabular}{lllll} \textbf{File size} & \textbf{NAND Page size} & -$\bf R_{DCP}$ & $\bf R_{DCP2}$ +$\bf R_{FCP}$ & $\bf R_{FCP2}$ & \textbf{RAM required}\\ \hline 64 MiB & 512 bytes & 256 KiB & 8 MiB & 32 bytes\\ @@ -510,10 +509,9 @@ \end{tabular} \end{center} -Besides $R_{DCP2}$ JFFS3 defines one more constant $R_{DCP2}^{min}$ +Beside $R_{FCP2}$, JFFS3 defines one more constant $R_{FCP2}^{min}$ denoting the minimal size of file which can have an -associated DCP2 node. - +associated FCP2 node. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % @@ -933,8 +931,8 @@ \item \emph{BB} - Bad Block \item \emph{CRC} - Cyclic Redundancy Check \item \emph{CP} - Check-Point -\item \emph{DCP} - Data Check-Point -\item \emph{DCP2} - Data Check-Point level 2 +\item \emph{FCP} - File Check-Point +\item \emph{FCP2} - File Check-Point level 2 \item \emph{JFFS2} - Journalling Flash File System version 2 \item \emph{JFFS3} - Journalling Flash File System version 3 \item \emph{MTD} - Memory Technology Devices From dedekind at infradead.org Sat Apr 23 13:46:28 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:49 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.46,1.47 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv30426 Modified Files: JFFS3design.tex Log Message: Start Directory checkpoint chapter Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.46 retrieving revision 1.47 diff -u -r1.46 -r1.47 --- JFFS3design.tex 23 Apr 2005 17:00:04 -0000 1.46 +++ JFFS3design.tex 23 Apr 2005 17:46:24 -0000 1.47 @@ -360,17 +360,17 @@ The data structure corresponding to FCP is: \begin{verbatim} -struct jffs3_raw_file_checkpoint +struct jffs3_raw_fcp { uint16_t magic; /* Magic bitmask of FCP node. */ + uint64_t version; /* Helps to differentiate between valid uint16_t index; /* The FCP index. Gives FCP range offset if multiplied by the FCP range. */ - uint64_t version; /* Helps to differentiate between valid - and obsolete data checkpoints. */ + and obsolete file checkpoints. */ uint32_t hdr_crc; /* FCP Header CRC32 checksum. */ - uint32_t crc; /* FCP CRC32 checksum. */ + uint32_t crc; /* FCP payload CRC32 checksum. */ - /* An array of references to nodes corresponding to the + /* An array of references to data nodes corresponding to * FCP. The array is sorted by the node range offset in * ascending order. */ struct jffs3_fcp_entry entries[]; @@ -513,6 +513,45 @@ denoting the minimal size of file which can have an associated FCP2 node. +\subsection{Directory checkpoints} +Directory checkpoints (DCP): +\begin{itemize} +\item are associated with directory inodes; +\item refer all the valid directory entries of the directory; +\item allow to quickly find the directory entry nodes by their names. +\end{itemize} + +The DCP data structure looks as follows. + +\begin{verbatim} +struct jffs3_raw_dcp +{ + uint16_t magic; /* Magic bitmask of DCP node. */ + uint64_t version; /* Helps to differentiate between valid + and obsolete directory checkpoints. */ + uint32_t hstart; /* Starting value of the DCP hash range. */ + uint32_t hend; /* Ending value of the DCP hash range. */ + uint32_t hdr_crc; /* DCP Header CRC32 checksum. */ + uint32_t crc; /* DCP payload CRC32 checksum. */ + + /* An array of references to direntry nodes corresponding to + * DCP. The array is sorted by the direntry name hash value + * in ascending order. */ + struct jffs3_dcp_entry entries[]; +} __attribute__((packed)); + +struct jffs3_dcp_entry +{ + uint32_t hash; + uint32_t offs; +} __attribute__((packed)); +\end{verbatim} + +To split large DCP on several nodes JFFS3 makes use of hash function +$H$ mapping directory entry names to 32-bit integers. In case of good +$H$ the probability of name collisions will be $1/2^{32}$. + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % VIRTUAL BLOCK HANDLING From dedekind at infradead.org Sun Apr 24 07:50:34 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:49 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.47,1.48 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv17477 Modified Files: JFFS3design.tex Log Message: Update Directory checkpoint chapter Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.47 retrieving revision 1.48 diff -u -r1.47 -r1.48 --- JFFS3design.tex 23 Apr 2005 17:46:24 -0000 1.47 +++ JFFS3design.tex 24 Apr 2005 11:50:31 -0000 1.48 @@ -15,6 +15,9 @@ \topmargin=0cm \textheight=24cm +% Define TODO command +\newcommand{\TODO}[1]{({\textbf{TODO}: #1})\marginpar{\large \textbf{!?!}}} + \begin{document} % @@ -199,10 +202,10 @@ entries in it) is browsed, more RAM is needed. \item[Peak RAM usage.] The JFFS2 memory consumption also depends on how data is -written. Each transaction goes directly to Flash (through the -\mbox{write-buffer} for \mbox{page-based} Flashes like NAND), i.e., there is +written. Each transaction goes directly to flash (through the +\mbox{write-buffer} for \mbox{page-based} flashes like NAND), i.e., there is \mbox{Write-Through} cache in Linux/JFFS2. Consequently, small transactions -result in a great deal of small nodes on Flash and hence, much memory is required, +result in a great deal of small nodes on flash and hence, much memory is required, for \mbox{in-core} objects and fragtree. Later small nodes will be merged by GC (maximum \texttt{PAGE\_SIZE} bytes of a files' data may be stored in one node) and the amount of consumed memory will be much lower. But the peak @@ -215,12 +218,12 @@ \subsection{Mount time} The slow mount is the most prominent and upsetting JFFS2 feature. The reason is, again, the JFFS2 design which doesn't assume any definite -structure on the Flash media but instead, makes use of one big log made +structure on the flash media but instead, makes use of one big log made up of nodes -- the only JFFS2 \mbox{on-flash} data structure. But unfortunately, there are several drawbacks, for example -- slow mount. -Because of absence of any Flash structure, JFFS2 needs to scan the whole -Flash partition to identify all nodes and to build the file system map. This -takes much time, especially on large Flash partitions. +Because of absence of any flash structure, JFFS2 needs to scan the whole +flash partition to identify all nodes and to build the file system map. This +takes much time, especially on large flash partitions. To increase the mount speed JFFS2 performs as few work as possible during the mount process and defers @@ -242,7 +245,7 @@ writers for a considerable time interval. \end{description} -TODO: show some real numbers here. +\TODO{show some real numbers here} The first problem may be solved by a patch referred to as a "summary patch" and accessible at \url{http://www.inf.u-szeged.hu/jffs2/}. @@ -266,7 +269,7 @@ build time is the linear function of the nodes number. \end{description} -TODO: provide some real numbers here +\TODO{provide some real numbers here} \subsection{Conclusion} Historically, JFFS2 was designed for small NOR flash chips and didn't even @@ -278,7 +281,7 @@ space as it actually need, without wasting much space as in case of "traditional" file systems for block devices; \item admitting of very efficient utilizing of "on-flight" compression which -allows to fit a big deal of data to Flash; +allows to fit a big deal of data to flash; \item quick read and write operations; \item natural unclean reboot robustness; \item good wear-leveling. @@ -298,7 +301,7 @@ \begin{enumerate} \item Fully POSIX-compatible file system. -\item The main OS is Linux (TODO: nonetheless portable?). +\item The main OS is Linux. \TODO{nonetheless portable?} \item Good scalability. \item Preserve the possibility to efficiently use compression (like in JFFS2). \item Provide good wear-leveling. @@ -306,7 +309,7 @@ \item Fast mounting. \item Reasonably low memory consumption. \item Fast enough read/write operations. -\item Support different Flash types (NOR, NAND, etc). +\item Support different flash types (NOR, NAND, etc). \item Support xattr. \end{enumerate} @@ -330,7 +333,7 @@ \end{itemize} % -% Data checkpoints +% File checkpoints % \subsection{File checkpoints} File checkpoints (FCP): @@ -378,7 +381,7 @@ struct jffs3_raw_fcp_entry { - uint32_t phys_offs; /* The position of the node on Flash. */ + uint32_t phys_offs; /* The position of the node on flash. */ uint32_t offs; /* Offset of the data range the node refers. */ uint16_t len; /* The length of the node data range. */ } __attribute__((packed)); @@ -404,14 +407,14 @@ \item[FCP node size.] FCP node physical size shouldn't be neither too large nor too small. To facilitate faster FCP update on NAND -flashes the maximum FCP node physical size ($S_{FCP}$) should be one NAND -page (or less). +flashes the maximum FCP node physical size ($S_{FCP}$) should be one +flash page (or less). Since one FCP entry takes 10 bytes, the FCP node header takes 20 bytes the maximum number of FCP entries $E_{FCP}$ will be: \begin{center} \begin{tabular}{ll} -\textbf{NAND page size} & $\bf E_{FCP}$ \\ +\textbf{Flash page size} & $\bf E_{FCP}$ \\ \hline 512 bytes & 49\\ 2048 bytes & 202\\ @@ -426,9 +429,9 @@ $$R_{FCP} > \mathtt{PAGE\_SIZE}{\cdot}E_{FCP},$$ where $E_{FCP}$ is the maximal number of FCP entries in FCP. The table below illustrates -the average data node range size depending on $R$ and NAND page size. +the average data node range size depending on $R$ and flash page size. -512 bytes NAND page: +512 bytes flash page: \begin{center} \begin{tabular}{ll} $\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ @@ -438,7 +441,7 @@ \end{tabular} \end{center} -2048 bytes NAND page: +2048 bytes flash page: \begin{center} \begin{tabular}{ll} $\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ @@ -449,7 +452,7 @@ \end{tabular} \end{center} -4096 bytes NAND page: +4096 bytes flash page: \begin{center} \begin{tabular}{ll} $\bf R_{FCP}$ & \textbf{Average $\bf R_{data}$}\\ @@ -460,7 +463,7 @@ \end{tabular} \end{center} -The above tables assumei that each FCP entry may at most refer 4~KiB of data. +The above tables assumes that each FCP entry may at most refer 4~KiB of data. For NOR and other non-paged flashes JFFS3 just assumes that the flash page size is virtually 512 bytes or larger, depending on the @@ -468,22 +471,22 @@ \end{description} If we keep in-core references of all the FCP nodes of a file, we may -waste a lot ot RAM. Even assuming that each reference takes 4~bytes +waste a lot of RAM. Even assuming that each reference takes 4~bytes (which is in fact unreachably few) we have: \begin{center} \begin{tabular}{lllll} -\textbf{File size} & \textbf{NAND Page size} & +\textbf{File size} & \textbf{Flash Page size} & $\bf R_{FCP}$ & \textbf{RAM required} & \textbf{Flash overhead}\\ \hline 64 MiB & 512 bytes & 128 KiB & 2 MiB & 256 KiB (0.4\%)\\ 1 GiB & 2048 bytes & 512 KiB & 8 MiB & 4 MiB (0.4\%)\\ -8 GiB & 4096 bytes & 1 MiB & 32 MiB & 16 MiB (0.4\%)\\ +4 GiB & 4096 bytes & 1 MiB & 16 MiB & 8 MiB (0.4\%)\\ \end{tabular} \end{center} -The above table also shows the amount of Flash space required to store -FCP nodes (assuming each FCP entry takes the whole Flash page). +The above table also shows the amount of flash space required to store +FCP nodes (assuming each FCP entry takes the whole flash page). To relax RAM usage JFFS3 makes use of \emph{second level file checkpoints} (FCP2) referring @@ -493,19 +496,19 @@ \begin{itemize} \item FCP2 range $R_{FCP2}$ is multiple of FCP range $R_{FCP}$; \item the largest FCP2 node physical size $S_{FCP2}$ is limited by the -NAND page size; +Flash page size; \end{itemize} With FCP2 the above table looks as: \begin{center} \begin{tabular}{lllll} -\textbf{File size} & \textbf{NAND Page size} & +\textbf{File size} & \textbf{Flash Page size} & $\bf R_{FCP}$ & $\bf R_{FCP2}$ & \textbf{RAM required}\\ \hline 64 MiB & 512 bytes & 256 KiB & 8 MiB & 32 bytes\\ 1 GiB & 2048 bytes & 1 MiB & 128 MiB & 32 bytes\\ -8 GiB & 4096 bytes & 2 MiB & 512 MiB & 64 bytes\\ +4 GiB & 4096 bytes & 1 MiB & 256 MiB & 64 bytes\\ \end{tabular} \end{center} @@ -513,6 +516,9 @@ denoting the minimal size of file which can have an associated FCP2 node. +% +% Directory checkpoints +% \subsection{Directory checkpoints} Directory checkpoints (DCP): \begin{itemize} @@ -529,8 +535,8 @@ uint16_t magic; /* Magic bitmask of DCP node. */ uint64_t version; /* Helps to differentiate between valid and obsolete directory checkpoints. */ - uint32_t hstart; /* Starting value of the DCP hash range. */ - uint32_t hend; /* Ending value of the DCP hash range. */ + uint32_t hstart; /* Starting value of the DCP hash diapason. */ + uint32_t hend; /* Ending value of the DCP hash diapason. */ uint32_t hdr_crc; /* DCP Header CRC32 checksum. */ uint32_t crc; /* DCP payload CRC32 checksum. */ @@ -551,6 +557,86 @@ $H$ mapping directory entry names to 32-bit integers. In case of good $H$ the probability of name collisions will be $1/2^{32}$. +It makes sense to use CRC32 as the hash function due to the following +reasons: +\begin{itemize} +\item CRC32 is certanly good hash function; +\item directory entry names on flash are protected by CRC32 and we +would calculate CRC32 anyway when we write the directory entry. +\end{itemize} + +To refer Directory checkpoints JFFS3 ought to keep in-core: +\begin{itemize} +\item DCP offset on flash; +\item Starting hash diapason value; +\item Ending hash diapason value; +\end{itemize} + +To find DCP which refers the direntry node with a given name JFFS3 +performs the following steps: +\begin{itemize} +\item calculates the hash value of the given name; +\item looks for DCP with the hash diapason that includes the calculated +hash value; +\item reads the DCP node and finds out the directory entry node offset; +\item reads the direntry node. +\end{itemize} + +Unlike FCP, directory checkpoints do not have any criteria to split them +on a fixed basis. Instead, DCP may refer any hash diapason, starting +from \mbox{\texttt{0x00000000}-\texttt{0xFFFFFFFF}} and ending with a +diapason including only one hash value (if there are too many +hash collisions). This means that JFFS3 dynamically splits and merges +DCP diapasons when new direntries are created and old direntries are +deleted or changed. + +The following constants are defined for directory checkpoints: +\begin{enumerate} +\item $E_{DCP}^{min}$ - the minimal number of DCP entries DCP may +refer; +\item $E_{DCP}^{max}$ - the maximal number of DCP entries DCP may refer; +\end{enumerate} + +DCP nodes physical size is restricted by the flash page size. Hence, +assuming $E_{DCP}^{min} = E_{DCP}^{max}/3$, +$E_{DCP}^{min}$ and $E_{DCP}^{max}$ for different flash page sizes are: + +\begin{center} +\begin{tabular}{lll} +\textbf{Flash page size} & $\bf E_{DCP}^{min}$ & $\bf E_{DCP}^{max}$\\ +\hline +512 bytes & 20 & 60\\ +2048 bytes & 84 & 252\\ +4096 bytes & 169 & 508\\ +\end{tabular} +\end{center} + +The JFFS3 DCP merging/splitting algorithm works as follows: +\begin{itemize} +\item as long as there are few direntries in directory (i.e less then +$E_{DCP}^{max}$), JFFS3 maintains only on DCP comprising whole +\mbox{32-bit} hash diapason; + +\item when more direntries are added, JFFS3 creates more directory +checkpoints splitting the hash diapason; + +\item JFFS3 inserts direntries to DCP with the correspondent hash +diapason; when the number of associated direntries of the DCP becomes +too large (i.e., greater then $E_{DCP}^{max}$), the DCP is split on two +directory checkpoints and the hash diapason is split correspondingly; + +\item when directory entries are deleted the DCP which correspond to +their diapasons becomes smaller; when DCP becomes too small (i.e., the +number of direntry nodes it refers becomes less then $E_{DCP}^{min}$) +JFFS3 either merges it with one of small neighbors or splits one of the +neighbors and then merges the two adjacent small diapasons. +\end{itemize} + +When several direntries have the colliding $H$ values, JFFS3 puts them +to the same diaposon. If the number of colliding direntries becomes +equvalent to $E_{DCP}^{max}$, JFFS3 will refuse adding one more direntry +with the colliding hash value. This implies there is a restriction on the number +of direntries in a directory. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % @@ -955,8 +1041,7 @@ \item each data node involves some additional flash space overhead because of its header; -\item JFFS2 needs more RAM to keep track of nodes; this is the worst -drawback, see (\ref{ref_JFFS2_and_mem}) for more information about this. +\item JFFS2 needs more RAM to keep track of nodes; \end{itemize} @@ -1135,9 +1220,8 @@ % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Credits} -David Woodhouse\\ Joern Engel\\ -Thomas Gleixner\\ +David Woodhouse\\ Josh Boyer\\ Ferenc Havasi\\ Joakim Tjernlund\\ From dedekind at infradead.org Tue Apr 26 11:18:06 2005 From: dedekind at infradead.org (Artem Bityuckiy) Date: Mon May 15 21:51:49 2006 Subject: mtd/fs/jffs3 JFFS3design.tex,1.48,1.49 Message-ID: Update of /home/cvs/mtd/fs/jffs3 In directory phoenix.infradead.org:/tmp/cvs-serv26850 Modified Files: JFFS3design.tex Log Message: Fix: KiB -> MiB Index: JFFS3design.tex =================================================================== RCS file: /home/cvs/mtd/fs/jffs3/JFFS3design.tex,v retrieving revision 1.48 retrieving revision 1.49 diff -u -r1.48 -r1.49 --- JFFS3design.tex 24 Apr 2005 11:50:31 -0000 1.48 +++ JFFS3design.tex 26 Apr 2005 15:18:03 -0000 1.49 @@ -162,9 +162,9 @@ \item[Experiment 2a (imaginary)]~ \begin{center} \begin{tabular}{ll} -jffs2\_node\_ref & 27.8 KiB \\ -jffs2\_node\_frag & 48.6 KiB \\ -jffs2\_full\_dnode & 27.8 KiB \\ +jffs2\_node\_ref & 27.8 MiB \\ +jffs2\_node\_frag & 48.6 MiB \\ +jffs2\_full\_dnode & 27.8 MiB \\ \hline total & 104.2 MiB \end{tabular} @@ -173,9 +173,9 @@ \item[Experiment 3a (imaginary)]~ \begin{center} \begin{tabular}{ll} -jffs2\_node\_ref & 198.5 KiB \\ -jffs2\_node\_frag & 346.9 KiB \\ -jffs2\_full\_dnode & 198.2 KiB \\ +jffs2\_node\_ref & 198.5 MiB \\ +jffs2\_node\_frag & 346.9 MiB \\ +jffs2\_full\_dnode & 198.2 MiB \\ \hline total & 743.6 MiB \end{tabular} From tpoynor at infradead.org Wed Apr 27 16:01:52 2005 From: tpoynor at infradead.org (tpoynor@infradead.org) Date: Mon May 15 21:51:49 2006 Subject: mtd/drivers/mtd/chips cfi_cmdset_0001.c,1.175,1.176 Message-ID: Update of /home/cvs/mtd/drivers/mtd/chips In directory phoenix.infradead.org:/tmp/cvs-serv23243/drivers/mtd/chips Modified Files: cfi_cmdset_0001.c Log Message: DEBUG_LOCK_BITS fixes for Intel NOR flash: adjust chip-relative offsets to block address, write to block address + 2 per recent datasheets. Signed-off-by: Todd Poynor Index: cfi_cmdset_0001.c =================================================================== RCS file: /home/cvs/mtd/drivers/mtd/chips/cfi_cmdset_0001.c,v retrieving revision 1.175 retrieving revision 1.176 diff -u -r1.175 -r1.176 --- cfi_cmdset_0001.c 1 Apr 2005 16:36:25 -0000 1.175 +++ cfi_cmdset_0001.c 27 Apr 2005 20:01:49 -0000 1.176 @@ -1812,8 +1812,9 @@ struct cfi_private *cfi = map->fldrv_priv; int status, ofs_factor = cfi->interleave * cfi->device_type; + adr += chip->start; xip_disable(map, chip, adr+(2*ofs_factor)); - cfi_send_gen_cmd(0x90, 0x55, 0, map, cfi, cfi->device_type, NULL); + map_write(map, CMD(0x90), adr+(2*ofs_factor)); chip->state = FL_JEDEC_QUERY; status = cfi_read_query(map, adr+(2*ofs_factor)); xip_enable(map, chip, 0);