ext4: Issue with ext4write

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

ext4: Issue with ext4write

Aaron Williams
Hi all,

I am sometimes seeing issues when using ext4write where fsck later complains
that the group descriptor has an invalid number of unused inodes. Is this a
known problem?

Also, I think the assert(offset == sizeof(*desc)); in ext4fs_checksum_update()
is invalid since with ext4 the descriptor is larger than the offset. When
debugging was enabled I'd always hit this assert.

Also, in ext4fs_write, the debug statement should say blocks and not bytes.

-Aaron

--
Aaron Williams
Senior Software Engineer
Cavium, Inc.
(408) 943-7198 (510) 789-8988 (cell)
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Aaron Williams
Hi all,

It looks like after a certain amount of data has been written that all hell
breaks loose with the ext4 filesystem.

In my case, I have the following files on a 64G USB thumb drive with two
partitions, a small FAT partition with the rest of the space dedicated to ext4
created using mkfs.ext4 /dev/sdg2

total 477632
-rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
drwx------ 2 root root     16384 Jul 23 18:10 lost+found
-rwxr-xr-x 1 root root  90706976 Dec 31  1969 test.64
-rwxr-xr-x 1 root root 152777216 Dec 31  1969 test.img
-rwxr-xr-x 1 root root        50 Dec 31  1969 test.txt
-rwxr-xr-x 1 root root   1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
-rwxr-xr-x 1 root root  90706976 Jul 23 18:11 vmlinux.64

Everything is fine until I wrote the file test.64, which is basically a copy
of vmlinux.64.

The first few files were written using my host Linux system, namely Image, u-
boot-octeon_ebb7304.bin and vmlinux.64.

From within U-Boot I performed the following commands:

# ext4load usb 0:2 $loadaddr Image
# ext4write usb 0:2 $fileaddr /test.img $filesize
# tftpboot $loadaddr test.txt
# ext4write usb 0:2 $fileaddr /test.txt $filesize
# ext4load usb 0:2 $loadaddr vmlinux.64
# ext4write usb 0:2 $fileaddr /test.64 $filesize

Everything is fine on the drive until I write test.64. At this point, I get a
huge list of errors:

# fsck.ext4 -v -n /dev/sdg2
e2fsck 1.42.11 (09-Jul-2014)
Group descriptor 0 has invalid unused inodes count 57374.  Fix? no

/dev/sdg2 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Deleted inode 130913 has zero dtime.  Fix? no

Inode 130915 is in use, but has dtime set.  Fix? no

Inode 130915 has imagic flag set.  Clear? no

Inode 130915 has a extra size (8223) which is invalid
Fix? no

Inode 130916 is in use, but has dtime set.  Fix? no

Inode 130916 has imagic flag set.  Clear? no

Inode 130916 has a extra size (8223) which is invalid
Fix? no

...
Illegal block #11 (2435760161) in inode 131070.  IGNORED.
Illegal indirect block (4026556192) in inode 131070.  IGNORED.
Illegal double indirect block (2433138721) in inode 131070.  IGNORED.
Illegal triple indirect block (2434195456) in inode 131070.  IGNORED.
Error while iterating over blocks in inode 131070: Illegal triply indirect
block found


and many many more errors.

Note that I am using the very latest ext4 code from the master branch.  This
is not a USB problem because I can reproduce this problem with an SD card.  
This problem also occurs on two different platforms, one being aarch64 little
endian and the other being MIPS64 big endian.  The filesystem code is
identical since the latest code has been backported to our older MIPS
bootloader.

My guess is that all hell is breaking loose when a file spans multiple block
groups.

-Aaron Williams

On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
>
> Hi all,
>
> I am sometimes seeing issues when using ext4write where fsck later
> complains
 that the group descriptor has an invalid number of unused
> inodes. Is this a known problem?
>
> Also, I think the assert(offset == sizeof(*desc)); in
> ext4fs_checksum_update()
 is invalid since with ext4 the descriptor is

> larger than the offset. When debugging was enabled I'd always hit this
> assert.
>
> Also, in ext4fs_write, the debug statement should say blocks and not bytes.
>
> -Aaron
>
> --
> Aaron Williams
> Senior Software Engineer
> Cavium, Inc.
> (408) 943-7198  (510) 789-8988 (cell)
> _______________________________________________
> U-Boot mailing list
> [hidden email]
> https://lists.denx.de/listinfo/u-boot

--
Aaron Williams
Senior Software Engineer
Cavium, Inc.
(408) 943-7198 (510) 789-8988 (cell)
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Aaron Williams
On Monday, July 23, 2018 11:15:59 PM PDT Aaron Williams wrote:

> External Email
>
> Hi all,
>
> It looks like after a certain amount of data has been written that all hell
> breaks loose with the ext4 filesystem.
>
> In my case, I have the following files on a 64G USB thumb drive with two
> partitions, a small FAT partition with the rest of the space dedicated to
> ext4
 created using mkfs.ext4 /dev/sdg2

>
> total 477632
> -rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
> drwx------ 2 root root     16384 Jul 23 18:10 lost+found
> -rwxr-xr-x 1 root root  90706976 Dec 31  1969 test.64
> -rwxr-xr-x 1 root root 152777216 Dec 31  1969 test.img
> -rwxr-xr-x 1 root root        50 Dec 31  1969 test.txt
> -rwxr-xr-x 1 root root   1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
> -rwxr-xr-x 1 root root  90706976 Jul 23 18:11 vmlinux.64
>
> Everything is fine until I wrote the file test.64, which is basically a
> copy
 of vmlinux.64.
>
> The first few files were written using my host Linux system, namely Image,
> u-
 boot-octeon_ebb7304.bin and vmlinux.64.

>
> From within U-Boot I performed the following commands:
>
> # ext4load usb 0:2 $loadaddr Image
> # ext4write usb 0:2 $fileaddr /test.img $filesize
> # tftpboot $loadaddr test.txt
> # ext4write usb 0:2 $fileaddr /test.txt $filesize
> # ext4load usb 0:2 $loadaddr vmlinux.64
> # ext4write usb 0:2 $fileaddr /test.64 $filesize
>
> Everything is fine on the drive until I write test.64. At this point, I get
> a
 huge list of errors:

>
> # fsck.ext4 -v -n /dev/sdg2
> e2fsck 1.42.11 (09-Jul-2014)
> Group descriptor 0 has invalid unused inodes count 57374.  Fix? no
>
> /dev/sdg2 contains a file system with errors, check forced.
> Pass 1: Checking inodes, blocks, and sizes
> Deleted inode 130913 has zero dtime.  Fix? no
>
> Inode 130915 is in use, but has dtime set.  Fix? no
>
> Inode 130915 has imagic flag set.  Clear? no
>
> Inode 130915 has a extra size (8223) which is invalid
> Fix? no
>
> Inode 130916 is in use, but has dtime set.  Fix? no
>
> Inode 130916 has imagic flag set.  Clear? no
>
> Inode 130916 has a extra size (8223) which is invalid
> Fix? no
>
> ...
> Illegal block #11 (2435760161) in inode 131070.  IGNORED.
> Illegal indirect block (4026556192) in inode 131070.  IGNORED.
> Illegal double indirect block (2433138721) in inode 131070.  IGNORED.
> Illegal triple indirect block (2434195456) in inode 131070.  IGNORED.
> Error while iterating over blocks in inode 131070: Illegal triply indirect
> block found
>
>
> and many many more errors.
>
> Note that I am using the very latest ext4 code from the master branch.
> This
 is not a USB problem because I can reproduce this problem with an SD
> card. This problem also occurs on two different platforms, one being
> aarch64 little endian and the other being MIPS64 big endian.  The
> filesystem code is identical since the latest code has been backported to
> our older MIPS bootloader.
>
> My guess is that all hell is breaking loose when a file spans multiple
> block
 groups.

>
> -Aaron Williams
>
> On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
>
> >
> >
> > Hi all,
> >
> >
> >
> > I am sometimes seeing issues when using ext4write where fsck later
> > complains
>
>  that the group descriptor has an invalid number of unused
>
> > inodes. Is this a known problem?
> >
> >
> >
> > Also, I think the assert(offset == sizeof(*desc)); in
> > ext4fs_checksum_update()
>
>  is invalid since with ext4 the descriptor is
>
> > larger than the offset. When debugging was enabled I'd always hit this
> > assert.
> >
> >
> >
> > Also, in ext4fs_write, the debug statement should say blocks and not
> > bytes.
>
> >
> >
> > -Aaron
> >
> >
> >
> > --
> > Aaron Williams
> > Senior Software Engineer
> > Cavium, Inc.
> > (408) 943-7198  (510) 789-8988 (cell)
> > _______________________________________________
> > U-Boot mailing list
> > [hidden email]
> > https://lists.denx.de/listinfo/u-boot
>
>
> --
> Aaron Williams
> Senior Software Engineer
> Cavium, Inc.
> (408) 943-7198  (510) 789-8988 (cell)
> _______________________________________________
> U-Boot mailing list
> [hidden email]
> https://lists.denx.de/listinfo/u-boot

Here is the output after the write operation from fsck.ext4 -v /dev/sdg2

fsck.ext4 -v  /dev/sdg2
e2fsck 1.42.11 (09-Jul-2014)
Group descriptor 0 has invalid unused inodes count 57374.  Fix<y>? yes
Pass 1: Checking inodes, blocks, and sizes

Running additional passes to resolve blocks claimed by more than one inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 7: 98307 98308 98309 98310 98311 98312
98313 98314 98315 98316 98317 98318 98319 98320 98321 98322 98323 98324 98325
98326 98327 98328 98329 98330 98331 98332 98333 98334 98335 98336 98337 98338
98339 98340 98341 98342 98343 98344 98345 98346 98347 98348 98349 98350 98351
98352 98353 98354 98355 98356 98357 98358 98359 98360 98361 98362 98363 98364
98365 98366 98367 98368 98369 98370 98371 98372 98373 98374 98375 98376 98377
98378 98379 98380 98381 98382 98383 98384 98385 98386 98387 98388 98389 98390
98391 98392 98393 98394 98395 98396 98397 98398 98399 98400 98401 98402 98403
98404 98405 98406 98407 98408 98409 98410 98411 98412 98413 98414 98415 98416
98417 98418 98419 98420 98421 98422 98423 98424 98425 98426 98427 98428 98429
98430 98431 98432 98433 98434 98435 98436 98437 98438 98439 98440 98441 98442
98443 98444 98445 98446 98447 98448 98449 98450 98451 98452 98453 98454 98455
98456 98457 98458 98459 98460 98461 98462 98463 98464 98465 98466 98467 98468
98469 98470 98471 98472 98473 98474 98475 98476 98477 98478 98479 98480 98481
98482 98483 98484 98485 98486 98487 98488 98489 98490 98491 98492 98493 98494
98495 98496 98497 98498 98499 98500 98501 98502 98503 98504 98505 98506 98507
98508 98509 98510 98511 98512 98513 98514 98515 98516 98517 98518 98519 98520
98521 98522 98523 98524 98525 98526 98527 98528 98529 98530 98531 98532 98533
98534 98535 98536 98537 98538 98539 98540 98541 98542 98543 98544 98545 98546
98547 98548 98549 98550 98551 98552 98553 98554 98555 98556 98557 98558 98559
98560 98561 98562 98563 98564 98565 98566 98567 98568 98569 98570 98571 98572
98573 98574 98575 98576 98577 98578 98579 98580 98581 98582 98583 98584 98585
98586 98587 98588 98589 98590 98591 98592 98593 98594 98595 98596 98597 98598
98599 98600 98601 98602 98603 98604 98605 98606 98607 98608 98609 98610 98611
98612 98613 98614 98615 98616 98617 98618 98619 98620 98621 98622 98623 98624
98625 98626 98627 98628 98629 98630 98631 98632 98633 98634 98635 98636 98637
98638 98639 98640 98641 98642 98643 98644 98645 98646 98647 98648 98649 98650
98651 98652 98653 98654 98655 98656 98657 98658 98659 98660 98661 98662 98663
98664 98665 98666 98667 98668 98669 98670 98671 98672 98673 98674 98675 98676
98677 98678 98679 98680 98681 98682 98683 98684 98685 98686 98687 98688 98689
98690 98691 98692 98693 98694 98695 98696 98697 98698 98699 98700 98701 98702
98703 98704 98705 98706 98707 98708 98709 98710 98711 98712 98713 98714 98715
98716 98717 98718 98719 98720 98721 98722 98723 98724 98725 98726 98727 98728
98729 98730 98731 98732 98733 98734 98735 98736 98737 98738 98739 98740 98741
98742 98743 98744 98745 98746 98747 98748 98749 98750 98751 98752 98753 98754
98755 98756 98757 98758 98759 98760 98761 98762 98763 98764 98765 98766 98767
98768 98769 98770 98771 98772 98773 98774 98775 98776 98777 98778 98779 98780
98781 98782 98783 98784 98785 98786 98787 98788 98789 98790 98791 98792 98793
98794 98795 98796 98797 98798 98799 98800 98801 98802 98803 98804 98805 98806
98807 98808 98809 98810 98811 98812 98813 98814 98815 98816 98817 98818 98819
98820 98821 98822 98823 98824 98825 98826 98827 98828 98829 98830 98831 98832
98833 98834 98835 98836 98837 98838 98839 98840 98841 98842 98843 98844 98845
98846 98847 98848 98849 98850 98851 98852 98853 98854 98855 98856 98857 98858
98859 98860 98861 98862 98863 98864 98865 98866 98867 98868 98869 98870 98871
98872 98873 98874 98875 98876 98877 98878 98879 98880 98881 98882 98883 98884
98885 98886 98887 98888 98889 98890 98891 98892 98893 98894 98895 98896 98897
98898 98899 98900 98901 98902 98903 98904 98905 98906 98907 98908 98909 98910
98911 98912 98913 98914 98915 98916 98917 98918 98919 98920 98921 98922 98923
98924 98925 98926 98927 98928 98929 98930 98931 98932 98933 98934 98935 98936
98937 98938 98939 98940 98941 98942 98943 98944 98945 98946 98947 98948 98949
98950 98951 98952 98953 98954 98955 98956 98957 98958 98959 98960 98961 98962
98963 98964 98965 98966 98967 98968 98969 98970 98971 98972 98973 98974 98975
98976 98977 98978 98979 98980 98981 98982 98983 98984 98985 98986 98987 98988
98989 98990 98991 98992 98993 98994 98995 98996 98997 98998 98999 99000 99001
99002 99003 99004 99005 99006 99007 99008 99009 99010 99011 99012 99013 99014
99015 99016 99017 99018 99019 99020 99021 99022 99023 99024 99025 99026 99027
99028 99029 99030 99031 99032 99033 99034 99035 99036 99037 99038 99039 99040
99041 99042 99043 99044 99045 99046 99047 99048 99049 99050 99051 99052 99053
99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066
99067 99068 99069 99070 99071 99072 99073 99074 99075 99076 99077 99078 99079
99080 99081 99082 99083 99084 99085 99086 99087 99088 99089 99090 99091 99092
99093 99094 99095 99096 99097 99098 99099 99100 99101 99102 99103 99104 99105
99106 99107 99108 99109 99110 99111 99112 99113 99114 99115 99116 99117 99118
99119 99120 99121 99122 99123 99124 99125 99126 99127 99128 99129 99130 99131
99132 99133 99134 99135 99136 99137 99138 99139 99140 99141 99142 99143 99144
99145 99146 99147 99148 99149 99150 99151 99152 99153 99154 99155 99156 99157
99158 99159 99160 99161 99162 99163 99164 99165 99166 99167 99168 99169 99170
99171 99172 99173 99174 99175 99176 99177 99178 99179 99180 99181 99182 99183
99184 99185 99186 99187 99188 99189 99190 99191 99192 99193 99194 99195 99196
99197 99198 99199 99200 99201 99202 99203 99204 99205 99206 99207 99208 99209
99210 99211 99212 99213 99214 99215 99216 99217 99218 99219 99220 99221 99222
99223 99224 99225 99226 99227 99228 99229 99230 99231 99232 99233 99234 99235
99236 99237 99238 99239 99240 99241 99242 99243 99244 99245 99246 99247 99248
99249 99250 99251 99252 99253 99254 99255 99256 99257 99258 99259 99260 99261
99262 99263 99264 99265 99266 99267 99268 99269 99270 99271 99272 99273 99274
99275 99276 99277 99278 99279 99280 99281 99282 99283 99284 99285 99286 99287
99288 99289 99290 99291 99292 99293 99294 99295 99296 99297 99298 99299 99300
99301 99302 99303 99304 99305 99306 99307 99308 99309 99310 99311 99312 99313
99314 99315 99316 99317 99318 99319 99320 99321 99322 99323 99324 99325 99326
99327 99328
Multiply-claimed block(s) in inode 15: 98304 98305 98306 98307 98308 98309
98310 98311 98312 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322
98323 98324 98325 98326 98327 98328 98329 98330 98331 98332 98333 98334 98335
98336 98337 98338 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348
98349 98350 98351 98352 98353 98354 98355 98356 98357 98358 98359 98360 98361
98362 98363 98364 98365 98366 98367 98368 98369 98370 98371 98372 98373 98374
98375 98376 98377 98378 98379 98380 98381 98382 98383 98384 98385 98386 98387
98388 98389 98390 98391 98392 98393 98394 98395 98396 98397 98398 98399 98400
98401 98402 98403 98404 98405 98406 98407 98408 98409 98410 98411 98412 98413
98414 98415 98416 98417 98418 98419 98420 98421 98422 98423 98424 98425 98426
98427 98428 98429 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439
98440 98441 98442 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452
98453 98454 98455 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465
98466 98467 98468 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478
98479 98480 98481 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491
98492 98493 98494 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504
98505 98506 98507 98508 98509 98510 98511 98512 98513 98514 98515 98516 98517
98518 98519 98520 98521 98522 98523 98524 98525 98526 98527 98528 98529 98530
98531 98532 98533 98534 98535 98536 98537 98538 98539 98540 98541 98542 98543
98544 98545 98546 98547 98548 98549 98550 98551 98552 98553 98554 98555 98556
98557 98558 98559 98560 98561 98562 98563 98564 98565 98566 98567 98568 98569
98570 98571 98572 98573 98574 98575 98576 98577 98578 98579 98580 98581 98582
98583 98584 98585 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595
98596 98597 98598 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608
98609 98610 98611 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621
98622 98623 98624 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634
98635 98636 98637 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647
98648 98649 98650 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660
98661 98662 98663 98664 98665 98666 98667 98668 98669 98670 98671 98672 98673
98674 98675 98676 98677 98678 98679 98680 98681 98682 98683 98684 98685 98686
98687 98688 98689 98690 98691 98692 98693 98694 98695 98696 98697 98698 98699
98700 98701 98702 98703 98704 98705 98706 98707 98708 98709 98710 98711 98712
98713 98714 98715 98716 98717 98718 98719 98720 98721 98722 98723 98724 98725
98726 98727 98728 98729 98730 98731 98732 98733 98734 98735 98736 98737 98738
98739 98740 98741 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751
98752 98753 98754 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764
98765 98766 98767 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777
98778 98779 98780 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790
98791 98792 98793 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803
98804 98805 98806 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816
98817 98818 98819 98820 98821 98822 98823 98824 98825 98826 98827 98828 98829
98830 98831 98832 98833 98834 98835 98836 98837 98838 98839 98840 98841 98842
98843 98844 98845 98846 98847 98848 98849 98850 98851 98852 98853 98854 98855
98856 98857 98858 98859 98860 98861 98862 98863 98864 98865 98866 98867 98868
98869 98870 98871 98872 98873 98874 98875 98876 98877 98878 98879 98880 98881
98882 98883 98884 98885 98886 98887 98888 98889 98890 98891 98892 98893 98894
98895 98896 98897 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907
98908 98909 98910 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920
98921 98922 98923 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933
98934 98935 98936 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946
98947 98948 98949 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959
98960 98961 98962 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972
98973 98974 98975 98976 98977 98978 98979 98980 98981 98982 98983 98984 98985
98986 98987 98988 98989 98990 98991 98992 98993 98994 98995 98996 98997 98998
98999 99000 99001 99002 99003 99004 99005 99006 99007 99008 99009 99010 99011
99012 99013 99014 99015 99016 99017 99018 99019 99020 99021 99022 99023 99024
99025 99026 99027 99028 99029 99030 99031 99032 99033 99034 99035 99036 99037
99038 99039 99040 99041 99042 99043 99044 99045 99046 99047 99048 99049 99050
99051 99052 99053 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063
99064 99065 99066 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076
99077 99078 99079 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089
99090 99091 99092 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102
99103 99104 99105 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115
99116 99117 99118 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128
99129 99130 99131 99132 99133 99134 99135 99136 99137 99138 99139 99140 99141
99142 99143 99144 99145 99146 99147 99148 99149 99150 99151 99152 99153 99154
99155 99156 99157 99158 99159 99160 99161 99162 99163 99164 99165 99166 99167
99168 99169 99170 99171 99172 99173 99174 99175 99176 99177 99178 99179 99180
99181 99182 99183 99184 99185 99186 99187 99188 99189 99190 99191 99192 99193
99194 99195 99196 99197 99198 99199 99200 99201 99202 99203 99204 99205 99206
99207 99208 99209 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219
99220 99221 99222 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232
99233 99234 99235 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245
99246 99247 99248 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258
99259 99260 99261 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271
99272 99273 99274 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284
99285 99286 99287 99288 99289 99290 99291 99292 99293 99294 99295 99296 99297
99298 99299 99300 99301 99302 99303 99304 99305 99306 99307 99308 99309 99310
99311 99312 99313 99314 99315 99316 99317 99318 99319 99320 99321 99322 99323
99324 99325 99326 99327 99328
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 1 inodes containing multiply-claimed blocks.)

File /test.img (inode #15, mod time Wed Dec 31 16:00:00 1969)
  has 1025 multiply-claimed block(s), shared with 2 file(s):
        <filesystem metadata>
        <The group descriptor inode> (inode #7, mod time Mon Jul 23 18:10:08
2018)
Clone multiply-claimed blocks<y>?
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sdg2: ***** FILE SYSTEM WAS MODIFIED *****

          17 inodes used (0.00%, out of 1913184)
           0 non-contiguous files (0.0%)
           0 non-contiguous directories (0.0%)
             # of inodes with ind/dind/tind blocks: 2/2/0
             Extent depth histogram: 6
      283495 blocks used (3.71%, out of 7640576)
           0 bad blocks
           1 large file

           6 regular files
           2 directories
           0 character device files
           0 block device files
           0 fifos
           0 links
           0 symbolic links (0 fast symbolic links)
           0 sockets
------------
           8 files

-Aaron

--
Aaron Williams
Senior Software Engineer
Cavium, Inc.
(408) 943-7198 (510) 789-8988 (cell)
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Marek Vasut
On 07/24/2018 08:25 AM, Aaron Williams wrote:

> On Monday, July 23, 2018 11:15:59 PM PDT Aaron Williams wrote:
>> External Email
>>
>> Hi all,
>>
>> It looks like after a certain amount of data has been written that all hell
>> breaks loose with the ext4 filesystem.
>>
>> In my case, I have the following files on a 64G USB thumb drive with two
>> partitions, a small FAT partition with the rest of the space dedicated to
>> ext4
>  created using mkfs.ext4 /dev/sdg2
>>
>> total 477632
>> -rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
>> drwx------ 2 root root     16384 Jul 23 18:10 lost+found
>> -rwxr-xr-x 1 root root  90706976 Dec 31  1969 test.64
>> -rwxr-xr-x 1 root root 152777216 Dec 31  1969 test.img
>> -rwxr-xr-x 1 root root        50 Dec 31  1969 test.txt
>> -rwxr-xr-x 1 root root   1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
>> -rwxr-xr-x 1 root root  90706976 Jul 23 18:11 vmlinux.64
>>
>> Everything is fine until I wrote the file test.64, which is basically a
>> copy
>  of vmlinux.64.
>>
>> The first few files were written using my host Linux system, namely Image,
>> u-
>  boot-octeon_ebb7304.bin and vmlinux.64.
>>
>> From within U-Boot I performed the following commands:
>>
>> # ext4load usb 0:2 $loadaddr Image
>> # ext4write usb 0:2 $fileaddr /test.img $filesize
>> # tftpboot $loadaddr test.txt
>> # ext4write usb 0:2 $fileaddr /test.txt $filesize
>> # ext4load usb 0:2 $loadaddr vmlinux.64
>> # ext4write usb 0:2 $fileaddr /test.64 $filesize
>>
>> Everything is fine on the drive until I write test.64. At this point, I get
>> a
>  huge list of errors:
>>
>> # fsck.ext4 -v -n /dev/sdg2
>> e2fsck 1.42.11 (09-Jul-2014)
>> Group descriptor 0 has invalid unused inodes count 57374.  Fix? no
>>
>> /dev/sdg2 contains a file system with errors, check forced.
>> Pass 1: Checking inodes, blocks, and sizes
>> Deleted inode 130913 has zero dtime.  Fix? no
>>
>> Inode 130915 is in use, but has dtime set.  Fix? no
>>
>> Inode 130915 has imagic flag set.  Clear? no
>>
>> Inode 130915 has a extra size (8223) which is invalid
>> Fix? no
>>
>> Inode 130916 is in use, but has dtime set.  Fix? no
>>
>> Inode 130916 has imagic flag set.  Clear? no
>>
>> Inode 130916 has a extra size (8223) which is invalid
>> Fix? no
>>
>> ...
>> Illegal block #11 (2435760161) in inode 131070.  IGNORED.
>> Illegal indirect block (4026556192) in inode 131070.  IGNORED.
>> Illegal double indirect block (2433138721) in inode 131070.  IGNORED.
>> Illegal triple indirect block (2434195456) in inode 131070.  IGNORED.
>> Error while iterating over blocks in inode 131070: Illegal triply indirect
>> block found
>>
>>
>> and many many more errors.
>>
>> Note that I am using the very latest ext4 code from the master branch.
>> This
>  is not a USB problem because I can reproduce this problem with an SD
>> card. This problem also occurs on two different platforms, one being
>> aarch64 little endian and the other being MIPS64 big endian.  The
>> filesystem code is identical since the latest code has been backported to
>> our older MIPS bootloader.
>>
>> My guess is that all hell is breaking loose when a file spans multiple
>> block
>  groups.
>>
>> -Aaron Williams
>>
>> On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
>>
>>>
>>>
>>> Hi all,
>>>
>>>
>>>
>>> I am sometimes seeing issues when using ext4write where fsck later
>>> complains
>>
>>  that the group descriptor has an invalid number of unused
>>
>>> inodes. Is this a known problem?
>>>
>>>
>>>
>>> Also, I think the assert(offset == sizeof(*desc)); in
>>> ext4fs_checksum_update()
>>
>>  is invalid since with ext4 the descriptor is
>>
>>> larger than the offset. When debugging was enabled I'd always hit this
>>> assert.
>>>
>>>
>>>
>>> Also, in ext4fs_write, the debug statement should say blocks and not
>>> bytes.
>>
>>>
>>>
>>> -Aaron
>>>
>>>
>>>
>>> --
>>> Aaron Williams
>>> Senior Software Engineer
>>> Cavium, Inc.
>>> (408) 943-7198  (510) 789-8988 (cell)
>>> _______________________________________________
>>> U-Boot mailing list
>>> [hidden email]
>>> https://lists.denx.de/listinfo/u-boot
>>
>>
>> --
>> Aaron Williams
>> Senior Software Engineer
>> Cavium, Inc.
>> (408) 943-7198  (510) 789-8988 (cell)
>> _______________________________________________
>> U-Boot mailing list
>> [hidden email]
>> https://lists.denx.de/listinfo/u-boot
>
> Here is the output after the write operation from fsck.ext4 -v /dev/sdg2
>
> fsck.ext4 -v  /dev/sdg2
> e2fsck 1.42.11 (09-Jul-2014)
> Group descriptor 0 has invalid unused inodes count 57374.  Fix<y>? yes
> Pass 1: Checking inodes, blocks, and sizes
>
> Running additional passes to resolve blocks claimed by more than one inode...
> Pass 1B: Rescanning for multiply-claimed blocks
> Multiply-claimed block(s) in inode 7: 98307 98308 98309 98310 98311 98312
> 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322 98323 98324 98325
> 98326 98327 98328 98329 98330 98331 98332 98333 98334 98335 98336 98337 98338
> 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348 98349 98350 98351
> 98352 98353 98354 98355 98356 98357 98358 98359 98360 98361 98362 98363 98364
> 98365 98366 98367 98368 98369 98370 98371 98372 98373 98374 98375 98376 98377
> 98378 98379 98380 98381 98382 98383 98384 98385 98386 98387 98388 98389 98390
> 98391 98392 98393 98394 98395 98396 98397 98398 98399 98400 98401 98402 98403
> 98404 98405 98406 98407 98408 98409 98410 98411 98412 98413 98414 98415 98416
> 98417 98418 98419 98420 98421 98422 98423 98424 98425 98426 98427 98428 98429
> 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439 98440 98441 98442
> 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452 98453 98454 98455
> 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465 98466 98467 98468
> 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478 98479 98480 98481
> 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491 98492 98493 98494
> 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504 98505 98506 98507
> 98508 98509 98510 98511 98512 98513 98514 98515 98516 98517 98518 98519 98520
> 98521 98522 98523 98524 98525 98526 98527 98528 98529 98530 98531 98532 98533
> 98534 98535 98536 98537 98538 98539 98540 98541 98542 98543 98544 98545 98546
> 98547 98548 98549 98550 98551 98552 98553 98554 98555 98556 98557 98558 98559
> 98560 98561 98562 98563 98564 98565 98566 98567 98568 98569 98570 98571 98572
> 98573 98574 98575 98576 98577 98578 98579 98580 98581 98582 98583 98584 98585
> 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595 98596 98597 98598
> 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608 98609 98610 98611
> 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621 98622 98623 98624
> 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634 98635 98636 98637
> 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647 98648 98649 98650
> 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660 98661 98662 98663
> 98664 98665 98666 98667 98668 98669 98670 98671 98672 98673 98674 98675 98676
> 98677 98678 98679 98680 98681 98682 98683 98684 98685 98686 98687 98688 98689
> 98690 98691 98692 98693 98694 98695 98696 98697 98698 98699 98700 98701 98702
> 98703 98704 98705 98706 98707 98708 98709 98710 98711 98712 98713 98714 98715
> 98716 98717 98718 98719 98720 98721 98722 98723 98724 98725 98726 98727 98728
> 98729 98730 98731 98732 98733 98734 98735 98736 98737 98738 98739 98740 98741
> 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751 98752 98753 98754
> 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764 98765 98766 98767
> 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777 98778 98779 98780
> 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790 98791 98792 98793
> 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803 98804 98805 98806
> 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816 98817 98818 98819
> 98820 98821 98822 98823 98824 98825 98826 98827 98828 98829 98830 98831 98832
> 98833 98834 98835 98836 98837 98838 98839 98840 98841 98842 98843 98844 98845
> 98846 98847 98848 98849 98850 98851 98852 98853 98854 98855 98856 98857 98858
> 98859 98860 98861 98862 98863 98864 98865 98866 98867 98868 98869 98870 98871
> 98872 98873 98874 98875 98876 98877 98878 98879 98880 98881 98882 98883 98884
> 98885 98886 98887 98888 98889 98890 98891 98892 98893 98894 98895 98896 98897
> 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907 98908 98909 98910
> 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920 98921 98922 98923
> 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933 98934 98935 98936
> 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946 98947 98948 98949
> 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959 98960 98961 98962
> 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972 98973 98974 98975
> 98976 98977 98978 98979 98980 98981 98982 98983 98984 98985 98986 98987 98988
> 98989 98990 98991 98992 98993 98994 98995 98996 98997 98998 98999 99000 99001
> 99002 99003 99004 99005 99006 99007 99008 99009 99010 99011 99012 99013 99014
> 99015 99016 99017 99018 99019 99020 99021 99022 99023 99024 99025 99026 99027
> 99028 99029 99030 99031 99032 99033 99034 99035 99036 99037 99038 99039 99040
> 99041 99042 99043 99044 99045 99046 99047 99048 99049 99050 99051 99052 99053
> 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066
> 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076 99077 99078 99079
> 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089 99090 99091 99092
> 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102 99103 99104 99105
> 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115 99116 99117 99118
> 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128 99129 99130 99131
> 99132 99133 99134 99135 99136 99137 99138 99139 99140 99141 99142 99143 99144
> 99145 99146 99147 99148 99149 99150 99151 99152 99153 99154 99155 99156 99157
> 99158 99159 99160 99161 99162 99163 99164 99165 99166 99167 99168 99169 99170
> 99171 99172 99173 99174 99175 99176 99177 99178 99179 99180 99181 99182 99183
> 99184 99185 99186 99187 99188 99189 99190 99191 99192 99193 99194 99195 99196
> 99197 99198 99199 99200 99201 99202 99203 99204 99205 99206 99207 99208 99209
> 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219 99220 99221 99222
> 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232 99233 99234 99235
> 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245 99246 99247 99248
> 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258 99259 99260 99261
> 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271 99272 99273 99274
> 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284 99285 99286 99287
> 99288 99289 99290 99291 99292 99293 99294 99295 99296 99297 99298 99299 99300
> 99301 99302 99303 99304 99305 99306 99307 99308 99309 99310 99311 99312 99313
> 99314 99315 99316 99317 99318 99319 99320 99321 99322 99323 99324 99325 99326
> 99327 99328
> Multiply-claimed block(s) in inode 15: 98304 98305 98306 98307 98308 98309
> 98310 98311 98312 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322
> 98323 98324 98325 98326 98327 98328 98329 98330 98331 98332 98333 98334 98335
> 98336 98337 98338 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348
> 98349 98350 98351 98352 98353 98354 98355 98356 98357 98358 98359 98360 98361
> 98362 98363 98364 98365 98366 98367 98368 98369 98370 98371 98372 98373 98374
> 98375 98376 98377 98378 98379 98380 98381 98382 98383 98384 98385 98386 98387
> 98388 98389 98390 98391 98392 98393 98394 98395 98396 98397 98398 98399 98400
> 98401 98402 98403 98404 98405 98406 98407 98408 98409 98410 98411 98412 98413
> 98414 98415 98416 98417 98418 98419 98420 98421 98422 98423 98424 98425 98426
> 98427 98428 98429 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439
> 98440 98441 98442 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452
> 98453 98454 98455 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465
> 98466 98467 98468 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478
> 98479 98480 98481 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491
> 98492 98493 98494 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504
> 98505 98506 98507 98508 98509 98510 98511 98512 98513 98514 98515 98516 98517
> 98518 98519 98520 98521 98522 98523 98524 98525 98526 98527 98528 98529 98530
> 98531 98532 98533 98534 98535 98536 98537 98538 98539 98540 98541 98542 98543
> 98544 98545 98546 98547 98548 98549 98550 98551 98552 98553 98554 98555 98556
> 98557 98558 98559 98560 98561 98562 98563 98564 98565 98566 98567 98568 98569
> 98570 98571 98572 98573 98574 98575 98576 98577 98578 98579 98580 98581 98582
> 98583 98584 98585 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595
> 98596 98597 98598 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608
> 98609 98610 98611 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621
> 98622 98623 98624 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634
> 98635 98636 98637 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647
> 98648 98649 98650 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660
> 98661 98662 98663 98664 98665 98666 98667 98668 98669 98670 98671 98672 98673
> 98674 98675 98676 98677 98678 98679 98680 98681 98682 98683 98684 98685 98686
> 98687 98688 98689 98690 98691 98692 98693 98694 98695 98696 98697 98698 98699
> 98700 98701 98702 98703 98704 98705 98706 98707 98708 98709 98710 98711 98712
> 98713 98714 98715 98716 98717 98718 98719 98720 98721 98722 98723 98724 98725
> 98726 98727 98728 98729 98730 98731 98732 98733 98734 98735 98736 98737 98738
> 98739 98740 98741 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751
> 98752 98753 98754 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764
> 98765 98766 98767 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777
> 98778 98779 98780 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790
> 98791 98792 98793 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803
> 98804 98805 98806 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816
> 98817 98818 98819 98820 98821 98822 98823 98824 98825 98826 98827 98828 98829
> 98830 98831 98832 98833 98834 98835 98836 98837 98838 98839 98840 98841 98842
> 98843 98844 98845 98846 98847 98848 98849 98850 98851 98852 98853 98854 98855
> 98856 98857 98858 98859 98860 98861 98862 98863 98864 98865 98866 98867 98868
> 98869 98870 98871 98872 98873 98874 98875 98876 98877 98878 98879 98880 98881
> 98882 98883 98884 98885 98886 98887 98888 98889 98890 98891 98892 98893 98894
> 98895 98896 98897 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907
> 98908 98909 98910 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920
> 98921 98922 98923 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933
> 98934 98935 98936 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946
> 98947 98948 98949 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959
> 98960 98961 98962 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972
> 98973 98974 98975 98976 98977 98978 98979 98980 98981 98982 98983 98984 98985
> 98986 98987 98988 98989 98990 98991 98992 98993 98994 98995 98996 98997 98998
> 98999 99000 99001 99002 99003 99004 99005 99006 99007 99008 99009 99010 99011
> 99012 99013 99014 99015 99016 99017 99018 99019 99020 99021 99022 99023 99024
> 99025 99026 99027 99028 99029 99030 99031 99032 99033 99034 99035 99036 99037
> 99038 99039 99040 99041 99042 99043 99044 99045 99046 99047 99048 99049 99050
> 99051 99052 99053 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063
> 99064 99065 99066 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076
> 99077 99078 99079 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089
> 99090 99091 99092 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102
> 99103 99104 99105 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115
> 99116 99117 99118 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128
> 99129 99130 99131 99132 99133 99134 99135 99136 99137 99138 99139 99140 99141
> 99142 99143 99144 99145 99146 99147 99148 99149 99150 99151 99152 99153 99154
> 99155 99156 99157 99158 99159 99160 99161 99162 99163 99164 99165 99166 99167
> 99168 99169 99170 99171 99172 99173 99174 99175 99176 99177 99178 99179 99180
> 99181 99182 99183 99184 99185 99186 99187 99188 99189 99190 99191 99192 99193
> 99194 99195 99196 99197 99198 99199 99200 99201 99202 99203 99204 99205 99206
> 99207 99208 99209 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219
> 99220 99221 99222 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232
> 99233 99234 99235 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245
> 99246 99247 99248 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258
> 99259 99260 99261 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271
> 99272 99273 99274 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284
> 99285 99286 99287 99288 99289 99290 99291 99292 99293 99294 99295 99296 99297
> 99298 99299 99300 99301 99302 99303 99304 99305 99306 99307 99308 99309 99310
> 99311 99312 99313 99314 99315 99316 99317 99318 99319 99320 99321 99322 99323
> 99324 99325 99326 99327 99328
> Pass 1C: Scanning directories for inodes with multiply-claimed blocks
> Pass 1D: Reconciling multiply-claimed blocks
> (There are 1 inodes containing multiply-claimed blocks.)
>
> File /test.img (inode #15, mod time Wed Dec 31 16:00:00 1969)
>   has 1025 multiply-claimed block(s), shared with 2 file(s):
>         <filesystem metadata>
>         <The group descriptor inode> (inode #7, mod time Mon Jul 23 18:10:08
> 2018)
> Clone multiply-claimed blocks<y>?
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
>
> /dev/sdg2: ***** FILE SYSTEM WAS MODIFIED *****
>
>           17 inodes used (0.00%, out of 1913184)
>            0 non-contiguous files (0.0%)
>            0 non-contiguous directories (0.0%)
>              # of inodes with ind/dind/tind blocks: 2/2/0
>              Extent depth histogram: 6
>       283495 blocks used (3.71%, out of 7640576)
>            0 bad blocks
>            1 large file
>
>            6 regular files
>            2 directories
>            0 character device files
>            0 block device files
>            0 fifos
>            0 links
>            0 symbolic links (0 fast symbolic links)
>            0 sockets
> ------------
>            8 files

I don't think mainline U-Boot supports octeon, but I might be wrong.
Which U-Boot version is this ? Can you replicate this on ie. sandbox?

--
Best regards,
Marek Vasut
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Aaron Williams
In reply to this post by Aaron Williams
On Monday, July 23, 2018 11:25:41 PM PDT Aaron Williams wrote:

> External Email
>
> On Monday, July 23, 2018 11:15:59 PM PDT Aaron Williams wrote:
>
> > External Email
> >
> >
> >
> > Hi all,
> >
> >
> >
> > It looks like after a certain amount of data has been written that all
> > hell
 breaks loose with the ext4 filesystem.

> >
> >
> >
> > In my case, I have the following files on a 64G USB thumb drive with two
> > partitions, a small FAT partition with the rest of the space dedicated to
> > ext4
>
>  created using mkfs.ext4 /dev/sdg2
>
> >
> >
> > total 477632
> > -rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
> > drwx------ 2 root root     16384 Jul 23 18:10 lost+found
> > -rwxr-xr-x 1 root root  90706976 Dec 31  1969 test.64
> > -rwxr-xr-x 1 root root 152777216 Dec 31  1969 test.img
> > -rwxr-xr-x 1 root root        50 Dec 31  1969 test.txt
> > -rwxr-xr-x 1 root root   1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
> > -rwxr-xr-x 1 root root  90706976 Jul 23 18:11 vmlinux.64
> >
> >
> >
> > Everything is fine until I wrote the file test.64, which is basically a
> > copy
>
>  of vmlinux.64.
>
> >
> >
> > The first few files were written using my host Linux system, namely
> > Image,
> > u-
>
>  boot-octeon_ebb7304.bin and vmlinux.64.
>
> >
> >
> > From within U-Boot I performed the following commands:
> >
> >
> >
> > # ext4load usb 0:2 $loadaddr Image
> > # ext4write usb 0:2 $fileaddr /test.img $filesize
> > # tftpboot $loadaddr test.txt
> > # ext4write usb 0:2 $fileaddr /test.txt $filesize
> > # ext4load usb 0:2 $loadaddr vmlinux.64
> > # ext4write usb 0:2 $fileaddr /test.64 $filesize
> >
> >
> >
> > Everything is fine on the drive until I write test.64. At this point, I
> > get
 a

>
>  huge list of errors:
>
> >
> >
> > # fsck.ext4 -v -n /dev/sdg2
> > e2fsck 1.42.11 (09-Jul-2014)
> > Group descriptor 0 has invalid unused inodes count 57374.  Fix? no
> >
> >
> >
> > /dev/sdg2 contains a file system with errors, check forced.
> > Pass 1: Checking inodes, blocks, and sizes
> > Deleted inode 130913 has zero dtime.  Fix? no
> >
> >
> >
> > Inode 130915 is in use, but has dtime set.  Fix? no
> >
> >
> >
> > Inode 130915 has imagic flag set.  Clear? no
> >
> >
> >
> > Inode 130915 has a extra size (8223) which is invalid
> > Fix? no
> >
> >
> >
> > Inode 130916 is in use, but has dtime set.  Fix? no
> >
> >
> >
> > Inode 130916 has imagic flag set.  Clear? no
> >
> >
> >
> > Inode 130916 has a extra size (8223) which is invalid
> > Fix? no
> >
> >
> >
> > ...
> > Illegal block #11 (2435760161) in inode 131070.  IGNORED.
> > Illegal indirect block (4026556192) in inode 131070.  IGNORED.
> > Illegal double indirect block (2433138721) in inode 131070.  IGNORED.
> > Illegal triple indirect block (2434195456) in inode 131070.  IGNORED.
> > Error while iterating over blocks in inode 131070: Illegal triply
> > indirect
> > block found
> >
> >
> >
> >
> > and many many more errors.
> >
> >
> >
> > Note that I am using the very latest ext4 code from the master branch.
> > This
>
>  is not a USB problem because I can reproduce this problem with an SD
>
> > card. This problem also occurs on two different platforms, one being
> > aarch64 little endian and the other being MIPS64 big endian.  The
> > filesystem code is identical since the latest code has been backported to
> > our older MIPS bootloader.
> >
> >
> >
> > My guess is that all hell is breaking loose when a file spans multiple
> > block
>
>  groups.
>
> >
> >
> > -Aaron Williams
> >
> >
> >
> > On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
> >
> >
> >
> > >
> > >
> > >
> > > Hi all,
> > >
> > >
> > >
> > >
> > >
> > > I am sometimes seeing issues when using ext4write where fsck later
> > > complains
> >
> >
> >
> >  that the group descriptor has an invalid number of unused
> >
> >
> >
> > > inodes. Is this a known problem?
> > >
> > >
> > >
> > >
> > >
> > > Also, I think the assert(offset == sizeof(*desc)); in
> > > ext4fs_checksum_update()
> >
> >
> >
> >  is invalid since with ext4 the descriptor is
> >
> >
> >
> > > larger than the offset. When debugging was enabled I'd always hit this
> > > assert.
> > >
> > >
> > >
> > >
> > >
> > > Also, in ext4fs_write, the debug statement should say blocks and not
> > > bytes.
> >
> >
> >
> > >
> > >
> > >
> > > -Aaron
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Aaron Williams
> > > Senior Software Engineer
> > > Cavium, Inc.
> > > (408) 943-7198  (510) 789-8988 (cell)
> > > _______________________________________________
> > > U-Boot mailing list
> > > [hidden email]
> > > https://lists.denx.de/listinfo/u-boot
> >
> >
> >
> >
> > --
> > Aaron Williams
> > Senior Software Engineer
> > Cavium, Inc.
> > (408) 943-7198  (510) 789-8988 (cell)
> > _______________________________________________
> > U-Boot mailing list
> > [hidden email]
> > https://lists.denx.de/listinfo/u-boot
>
>
> Here is the output after the write operation from fsck.ext4 -v /dev/sdg2
>
> fsck.ext4 -v  /dev/sdg2
> e2fsck 1.42.11 (09-Jul-2014)
> Group descriptor 0 has invalid unused inodes count 57374.  Fix<y>? yes
> Pass 1: Checking inodes, blocks, and sizes
>
> Running additional passes to resolve blocks claimed by more than one
> inode...
 Pass 1B: Rescanning for multiply-claimed blocks
> Multiply-claimed block(s) in inode 7: 98307 98308 98309 98310 98311 98312
> 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322 98323 98324
> 98325
 98326 98327 98328 98329 98330 98331 98332 98333 98334 98335 98336

> 98337 98338 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348
> 98349 98350 98351 98352 98353 98354 98355 98356 98357 98358 98359 98360
> 98361 98362 98363 98364 98365 98366 98367 98368 98369 98370 98371 98372
> 98373 98374 98375 98376 98377 98378 98379 98380 98381 98382 98383 98384
> 98385 98386 98387 98388 98389 98390 98391 98392 98393 98394 98395 98396
> 98397 98398 98399 98400 98401 98402 98403 98404 98405 98406 98407 98408
> 98409 98410 98411 98412 98413 98414 98415 98416 98417 98418 98419 98420
> 98421 98422 98423 98424 98425 98426 98427 98428 98429 98430 98431 98432
> 98433 98434 98435 98436 98437 98438 98439 98440 98441 98442 98443 98444
> 98445 98446 98447 98448 98449 98450 98451 98452 98453 98454 98455 98456
> 98457 98458 98459 98460 98461 98462 98463 98464 98465 98466 98467 98468
> 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478 98479 98480
> 98481 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491 98492
> 98493 98494 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504
> 98505 98506 98507 98508 98509 98510 98511 98512 98513 98514 98515 98516
> 98517 98518 98519 98520 98521 98522 98523 98524 98525 98526 98527 98528
> 98529 98530 98531 98532 98533 98534 98535 98536 98537 98538 98539 98540
> 98541 98542 98543 98544 98545 98546 98547 98548 98549 98550 98551 98552
> 98553 98554 98555 98556 98557 98558 98559 98560 98561 98562 98563 98564
> 98565 98566 98567 98568 98569 98570 98571 98572 98573 98574 98575 98576
> 98577 98578 98579 98580 98581 98582 98583 98584 98585 98586 98587 98588
> 98589 98590 98591 98592 98593 98594 98595 98596 98597 98598 98599 98600
> 98601 98602 98603 98604 98605 98606 98607 98608 98609 98610 98611 98612
> 98613 98614 98615 98616 98617 98618 98619 98620 98621 98622 98623 98624
> 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634 98635 98636
> 98637 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647 98648
> 98649 98650 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660
> 98661 98662 98663 98664 98665 98666 98667 98668 98669 98670 98671 98672
> 98673 98674 98675 98676 98677 98678 98679 98680 98681 98682 98683 98684
> 98685 98686 98687 98688 98689 98690 98691 98692 98693 98694 98695 98696
> 98697 98698 98699 98700 98701 98702 98703 98704 98705 98706 98707 98708
> 98709 98710 98711 98712 98713 98714 98715 98716 98717 98718 98719 98720
> 98721 98722 98723 98724 98725 98726 98727 98728 98729 98730 98731 98732
> 98733 98734 98735 98736 98737 98738 98739 98740 98741 98742 98743 98744
> 98745 98746 98747 98748 98749 98750 98751 98752 98753 98754 98755 98756
> 98757 98758 98759 98760 98761 98762 98763 98764 98765 98766 98767 98768
> 98769 98770 98771 98772 98773 98774 98775 98776 98777 98778 98779 98780
> 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790 98791 98792
> 98793 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803 98804
> 98805 98806 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816
> 98817 98818 98819 98820 98821 98822 98823 98824 98825 98826 98827 98828
> 98829 98830 98831 98832 98833 98834 98835 98836 98837 98838 98839 98840
> 98841 98842 98843 98844 98845 98846 98847 98848 98849 98850 98851 98852
> 98853 98854 98855 98856 98857 98858 98859 98860 98861 98862 98863 98864
> 98865 98866 98867 98868 98869 98870 98871 98872 98873 98874 98875 98876
> 98877 98878 98879 98880 98881 98882 98883 98884 98885 98886 98887 98888
> 98889 98890 98891 98892 98893 98894 98895 98896 98897 98898 98899 98900
> 98901 98902 98903 98904 98905 98906 98907 98908 98909 98910 98911 98912
> 98913 98914 98915 98916 98917 98918 98919 98920 98921 98922 98923 98924
> 98925 98926 98927 98928 98929 98930 98931 98932 98933 98934 98935 98936
> 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946 98947 98948
> 98949 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959 98960
> 98961 98962 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972
> 98973 98974 98975 98976 98977 98978 98979 98980 98981 98982 98983 98984
> 98985 98986 98987 98988 98989 98990 98991 98992 98993 98994 98995 98996
> 98997 98998 98999 99000 99001 99002 99003 99004 99005 99006 99007 99008
> 99009 99010 99011 99012 99013 99014 99015 99016 99017 99018 99019 99020
> 99021 99022 99023 99024 99025 99026 99027 99028 99029 99030 99031 99032
> 99033 99034 99035 99036 99037 99038 99039 99040 99041 99042 99043 99044
> 99045 99046 99047 99048 99049 99050 99051 99052 99053 99054 99055 99056
> 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066 99067 99068
> 99069 99070 99071 99072 99073 99074 99075 99076 99077 99078 99079 99080
> 99081 99082 99083 99084 99085 99086 99087 99088 99089 99090 99091 99092
> 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102 99103 99104
> 99105 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115 99116
> 99117 99118 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128
> 99129 99130 99131 99132 99133 99134 99135 99136 99137 99138 99139 99140
> 99141 99142 99143 99144 99145 99146 99147 99148 99149 99150 99151 99152
> 99153 99154 99155 99156 99157 99158 99159 99160 99161 99162 99163 99164
> 99165 99166 99167 99168 99169 99170 99171 99172 99173 99174 99175 99176
> 99177 99178 99179 99180 99181 99182 99183 99184 99185 99186 99187 99188
> 99189 99190 99191 99192 99193 99194 99195 99196 99197 99198 99199 99200
> 99201 99202 99203 99204 99205 99206 99207 99208 99209 99210 99211 99212
> 99213 99214 99215 99216 99217 99218 99219 99220 99221 99222 99223 99224
> 99225 99226 99227 99228 99229 99230 99231 99232 99233 99234 99235 99236
> 99237 99238 99239 99240 99241 99242 99243 99244 99245 99246 99247 99248
> 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258 99259 99260
> 99261 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271 99272
> 99273 99274 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284
> 99285 99286 99287 99288 99289 99290 99291 99292 99293 99294 99295 99296
> 99297 99298 99299 99300 99301 99302 99303 99304 99305 99306 99307 99308
> 99309 99310 99311 99312 99313 99314 99315 99316 99317 99318 99319 99320
> 99321 99322 99323 99324 99325 99326 99327 99328
> Multiply-claimed block(s) in inode 15: 98304 98305 98306 98307 98308 98309
> 98310 98311 98312 98313 98314 98315 98316 98317 98318 98319 98320 98321
> 98322
 98323 98324 98325 98326 98327 98328 98329 98330 98331 98332 98333

> 98334 98335 98336 98337 98338 98339 98340 98341 98342 98343 98344 98345
> 98346 98347 98348 98349 98350 98351 98352 98353 98354 98355 98356 98357
> 98358 98359 98360 98361 98362 98363 98364 98365 98366 98367 98368 98369
> 98370 98371 98372 98373 98374 98375 98376 98377 98378 98379 98380 98381
> 98382 98383 98384 98385 98386 98387 98388 98389 98390 98391 98392 98393
> 98394 98395 98396 98397 98398 98399 98400 98401 98402 98403 98404 98405
> 98406 98407 98408 98409 98410 98411 98412 98413 98414 98415 98416 98417
> 98418 98419 98420 98421 98422 98423 98424 98425 98426 98427 98428 98429
> 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439 98440 98441
> 98442 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452 98453
> 98454 98455 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465
> 98466 98467 98468 98469 98470 98471 98472 98473 98474 98475 98476 98477
> 98478 98479 98480 98481 98482 98483 98484 98485 98486 98487 98488 98489
> 98490 98491 98492 98493 98494 98495 98496 98497 98498 98499 98500 98501
> 98502 98503 98504 98505 98506 98507 98508 98509 98510 98511 98512 98513
> 98514 98515 98516 98517 98518 98519 98520 98521 98522 98523 98524 98525
> 98526 98527 98528 98529 98530 98531 98532 98533 98534 98535 98536 98537
> 98538 98539 98540 98541 98542 98543 98544 98545 98546 98547 98548 98549
> 98550 98551 98552 98553 98554 98555 98556 98557 98558 98559 98560 98561
> 98562 98563 98564 98565 98566 98567 98568 98569 98570 98571 98572 98573
> 98574 98575 98576 98577 98578 98579 98580 98581 98582 98583 98584 98585
> 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595 98596 98597
> 98598 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608 98609
> 98610 98611 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621
> 98622 98623 98624 98625 98626 98627 98628 98629 98630 98631 98632 98633
> 98634 98635 98636 98637 98638 98639 98640 98641 98642 98643 98644 98645
> 98646 98647 98648 98649 98650 98651 98652 98653 98654 98655 98656 98657
> 98658 98659 98660 98661 98662 98663 98664 98665 98666 98667 98668 98669
> 98670 98671 98672 98673 98674 98675 98676 98677 98678 98679 98680 98681
> 98682 98683 98684 98685 98686 98687 98688 98689 98690 98691 98692 98693
> 98694 98695 98696 98697 98698 98699 98700 98701 98702 98703 98704 98705
> 98706 98707 98708 98709 98710 98711 98712 98713 98714 98715 98716 98717
> 98718 98719 98720 98721 98722 98723 98724 98725 98726 98727 98728 98729
> 98730 98731 98732 98733 98734 98735 98736 98737 98738 98739 98740 98741
> 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751 98752 98753
> 98754 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764 98765
> 98766 98767 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777
> 98778 98779 98780 98781 98782 98783 98784 98785 98786 98787 98788 98789
> 98790 98791 98792 98793 98794 98795 98796 98797 98798 98799 98800 98801
> 98802 98803 98804 98805 98806 98807 98808 98809 98810 98811 98812 98813
> 98814 98815 98816 98817 98818 98819 98820 98821 98822 98823 98824 98825
> 98826 98827 98828 98829 98830 98831 98832 98833 98834 98835 98836 98837
> 98838 98839 98840 98841 98842 98843 98844 98845 98846 98847 98848 98849
> 98850 98851 98852 98853 98854 98855 98856 98857 98858 98859 98860 98861
> 98862 98863 98864 98865 98866 98867 98868 98869 98870 98871 98872 98873
> 98874 98875 98876 98877 98878 98879 98880 98881 98882 98883 98884 98885
> 98886 98887 98888 98889 98890 98891 98892 98893 98894 98895 98896 98897
> 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907 98908 98909
> 98910 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920 98921
> 98922 98923 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933
> 98934 98935 98936 98937 98938 98939 98940 98941 98942 98943 98944 98945
> 98946 98947 98948 98949 98950 98951 98952 98953 98954 98955 98956 98957
> 98958 98959 98960 98961 98962 98963 98964 98965 98966 98967 98968 98969
> 98970 98971 98972 98973 98974 98975 98976 98977 98978 98979 98980 98981
> 98982 98983 98984 98985 98986 98987 98988 98989 98990 98991 98992 98993
> 98994 98995 98996 98997 98998 98999 99000 99001 99002 99003 99004 99005
> 99006 99007 99008 99009 99010 99011 99012 99013 99014 99015 99016 99017
> 99018 99019 99020 99021 99022 99023 99024 99025 99026 99027 99028 99029
> 99030 99031 99032 99033 99034 99035 99036 99037 99038 99039 99040 99041
> 99042 99043 99044 99045 99046 99047 99048 99049 99050 99051 99052 99053
> 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065
> 99066 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076 99077
> 99078 99079 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089
> 99090 99091 99092 99093 99094 99095 99096 99097 99098 99099 99100 99101
> 99102 99103 99104 99105 99106 99107 99108 99109 99110 99111 99112 99113
> 99114 99115 99116 99117 99118 99119 99120 99121 99122 99123 99124 99125
> 99126 99127 99128 99129 99130 99131 99132 99133 99134 99135 99136 99137
> 99138 99139 99140 99141 99142 99143 99144 99145 99146 99147 99148 99149
> 99150 99151 99152 99153 99154 99155 99156 99157 99158 99159 99160 99161
> 99162 99163 99164 99165 99166 99167 99168 99169 99170 99171 99172 99173
> 99174 99175 99176 99177 99178 99179 99180 99181 99182 99183 99184 99185
> 99186 99187 99188 99189 99190 99191 99192 99193 99194 99195 99196 99197
> 99198 99199 99200 99201 99202 99203 99204 99205 99206 99207 99208 99209
> 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219 99220 99221
> 99222 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232 99233
> 99234 99235 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245
> 99246 99247 99248 99249 99250 99251 99252 99253 99254 99255 99256 99257
> 99258 99259 99260 99261 99262 99263 99264 99265 99266 99267 99268 99269
> 99270 99271 99272 99273 99274 99275 99276 99277 99278 99279 99280 99281
> 99282 99283 99284 99285 99286 99287 99288 99289 99290 99291 99292 99293
> 99294 99295 99296 99297 99298 99299 99300 99301 99302 99303 99304 99305
> 99306 99307 99308 99309 99310 99311 99312 99313 99314 99315 99316 99317
> 99318 99319 99320 99321 99322 99323 99324 99325 99326 99327 99328
> Pass 1C: Scanning directories for inodes with multiply-claimed blocks
> Pass 1D: Reconciling multiply-claimed blocks
> (There are 1 inodes containing multiply-claimed blocks.)
>
> File /test.img (inode #15, mod time Wed Dec 31 16:00:00 1969)
>   has 1025 multiply-claimed block(s), shared with 2 file(s):
>         <filesystem metadata>
>         <The group descriptor inode> (inode #7, mod time Mon Jul 23
> 18:10:08
 2018)

> Clone multiply-claimed blocks<y>?
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
>
> /dev/sdg2: ***** FILE SYSTEM WAS MODIFIED *****
>
>           17 inodes used (0.00%, out of 1913184)
>            0 non-contiguous files (0.0%)
>            0 non-contiguous directories (0.0%)
>              # of inodes with ind/dind/tind blocks: 2/2/0
>              Extent depth histogram: 6
>       283495 blocks used (3.71%, out of 7640576)
>            0 bad blocks
>            1 large file
>
>            6 regular files
>            2 directories
>            0 character device files
>            0 block device files
>            0 fifos
>            0 links
>            0 symbolic links (0 fast symbolic links)
>            0 sockets
> ------------
>            8 files
>
> -Aaron
>

I found another endian issue. In ext4fs_get_new_inode_no() it is setting bgd-
>bg_itable_unused without performing the proper endian conversion.
Additionally, if it's a 64-bit filesystem, it does not set
bg_itable_unused_high.

-Aaron

_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Aaron Williams
In reply to this post by Marek Vasut
On Tuesday, July 24, 2018 2:45:42 AM PDT Marek Vasut wrote:

> External Email
>
> On 07/24/2018 08:25 AM, Aaron Williams wrote:
> > On Monday, July 23, 2018 11:15:59 PM PDT Aaron Williams wrote:
> >> External Email
> >>
> >> Hi all,
> >>
> >> It looks like after a certain amount of data has been written that all
> >> hell
> >> breaks loose with the ext4 filesystem.
> >>
> >> In my case, I have the following files on a 64G USB thumb drive with two
> >> partitions, a small FAT partition with the rest of the space dedicated to
> >> ext4
> >>
> >  created using mkfs.ext4 /dev/sdg2
> >  
> >> total 477632
> >> -rwxr-xr-x 1 root root 152777216 Jul 23 18:11 Image
> >> drwx------ 2 root root     16384 Jul 23 18:10 lost+found
> >> -rwxr-xr-x 1 root root  90706976 Dec 31  1969 test.64
> >> -rwxr-xr-x 1 root root 152777216 Dec 31  1969 test.img
> >> -rwxr-xr-x 1 root root        50 Dec 31  1969 test.txt
> >> -rwxr-xr-x 1 root root   1841408 Jul 23 18:12 u-boot-octeon_ebb7304.bin
> >> -rwxr-xr-x 1 root root  90706976 Jul 23 18:11 vmlinux.64
> >>
> >> Everything is fine until I wrote the file test.64, which is basically a
> >> copy
> >>
> >  of vmlinux.64.
> >  
> >> The first few files were written using my host Linux system, namely
> >> Image,
> >> u-
> >>
> >  boot-octeon_ebb7304.bin and vmlinux.64.
> >  
> >> From within U-Boot I performed the following commands:
> >>
> >> # ext4load usb 0:2 $loadaddr Image
> >> # ext4write usb 0:2 $fileaddr /test.img $filesize
> >> # tftpboot $loadaddr test.txt
> >> # ext4write usb 0:2 $fileaddr /test.txt $filesize
> >> # ext4load usb 0:2 $loadaddr vmlinux.64
> >> # ext4write usb 0:2 $fileaddr /test.64 $filesize
> >>
> >> Everything is fine on the drive until I write test.64. At this point, I
> >> get
> >> a
> >>
> >  huge list of errors:
> >> # fsck.ext4 -v -n /dev/sdg2
> >> e2fsck 1.42.11 (09-Jul-2014)
> >> Group descriptor 0 has invalid unused inodes count 57374.  Fix? no
> >>
> >> /dev/sdg2 contains a file system with errors, check forced.
> >> Pass 1: Checking inodes, blocks, and sizes
> >> Deleted inode 130913 has zero dtime.  Fix? no
> >>
> >> Inode 130915 is in use, but has dtime set.  Fix? no
> >>
> >> Inode 130915 has imagic flag set.  Clear? no
> >>
> >> Inode 130915 has a extra size (8223) which is invalid
> >> Fix? no
> >>
> >> Inode 130916 is in use, but has dtime set.  Fix? no
> >>
> >> Inode 130916 has imagic flag set.  Clear? no
> >>
> >> Inode 130916 has a extra size (8223) which is invalid
> >> Fix? no
> >>
> >> ...
> >> Illegal block #11 (2435760161) in inode 131070.  IGNORED.
> >> Illegal indirect block (4026556192) in inode 131070.  IGNORED.
> >> Illegal double indirect block (2433138721) in inode 131070.  IGNORED.
> >> Illegal triple indirect block (2434195456) in inode 131070.  IGNORED.
> >> Error while iterating over blocks in inode 131070: Illegal triply
> >> indirect
> >> block found
> >>
> >>
> >> and many many more errors.
> >>
> >> Note that I am using the very latest ext4 code from the master branch.
> >> This
> >>
> >  is not a USB problem because I can reproduce this problem with an SD
> >  
> >> card. This problem also occurs on two different platforms, one being
> >> aarch64 little endian and the other being MIPS64 big endian.  The
> >> filesystem code is identical since the latest code has been backported to
> >> our older MIPS bootloader.
> >>
> >> My guess is that all hell is breaking loose when a file spans multiple
> >> block
> >>
> >  groups.
> >  
> >> -Aaron Williams
> >>
> >> On Thursday, July 19, 2018 7:35:46 PM PDT Aaron Williams wrote:
> >>> Hi all,
> >>>
> >>>
> >>>
> >>> I am sometimes seeing issues when using ext4write where fsck later
> >>> complains
> >>>
> >>  that the group descriptor has an invalid number of unused
> >>  
> >>> inodes. Is this a known problem?
> >>>
> >>>
> >>>
> >>> Also, I think the assert(offset == sizeof(*desc)); in
> >>> ext4fs_checksum_update()
> >>>
> >>  is invalid since with ext4 the descriptor is
> >>  
> >>> larger than the offset. When debugging was enabled I'd always hit this
> >>> assert.
> >>>
> >>>
> >>>
> >>> Also, in ext4fs_write, the debug statement should say blocks and not
> >>> bytes.
> >>>
> >>>
> >>>
> >>> -Aaron
> >>>
> >>>
> >>>
> >>> --
> >>> Aaron Williams
> >>> Senior Software Engineer
> >>> Cavium, Inc.
> >>> (408) 943-7198  (510) 789-8988 (cell)
> >>> _______________________________________________
> >>> U-Boot mailing list
> >>> [hidden email]
> >>> https://lists.denx.de/listinfo/u-boot
> >>
> >> --
> >> Aaron Williams
> >> Senior Software Engineer
> >> Cavium, Inc.
> >> (408) 943-7198  (510) 789-8988 (cell)
> >> _______________________________________________
> >> U-Boot mailing list
> >> [hidden email]
> >> https://lists.denx.de/listinfo/u-boot
> >
> > Here is the output after the write operation from fsck.ext4 -v /dev/sdg2
> >
> > fsck.ext4 -v  /dev/sdg2
> > e2fsck 1.42.11 (09-Jul-2014)
> > Group descriptor 0 has invalid unused inodes count 57374.  Fix<y>? yes
> > Pass 1: Checking inodes, blocks, and sizes
> >
> > Running additional passes to resolve blocks claimed by more than one
> > inode... Pass 1B: Rescanning for multiply-claimed blocks
> > Multiply-claimed block(s) in inode 7: 98307 98308 98309 98310 98311 98312
> > 98313 98314 98315 98316 98317 98318 98319 98320 98321 98322 98323 98324
> > 98325 98326 98327 98328 98329 98330 98331 98332 98333 98334 98335 98336
> > 98337 98338 98339 98340 98341 98342 98343 98344 98345 98346 98347 98348
> > 98349 98350 98351 98352 98353 98354 98355 98356 98357 98358 98359 98360
> > 98361 98362 98363 98364 98365 98366 98367 98368 98369 98370 98371 98372
> > 98373 98374 98375 98376 98377 98378 98379 98380 98381 98382 98383 98384
> > 98385 98386 98387 98388 98389 98390 98391 98392 98393 98394 98395 98396
> > 98397 98398 98399 98400 98401 98402 98403 98404 98405 98406 98407 98408
> > 98409 98410 98411 98412 98413 98414 98415 98416 98417 98418 98419 98420
> > 98421 98422 98423 98424 98425 98426 98427 98428 98429 98430 98431 98432
> > 98433 98434 98435 98436 98437 98438 98439 98440 98441 98442 98443 98444
> > 98445 98446 98447 98448 98449 98450 98451 98452 98453 98454 98455 98456
> > 98457 98458 98459 98460 98461 98462 98463 98464 98465 98466 98467 98468
> > 98469 98470 98471 98472 98473 98474 98475 98476 98477 98478 98479 98480
> > 98481 98482 98483 98484 98485 98486 98487 98488 98489 98490 98491 98492
> > 98493 98494 98495 98496 98497 98498 98499 98500 98501 98502 98503 98504
> > 98505 98506 98507 98508 98509 98510 98511 98512 98513 98514 98515 98516
> > 98517 98518 98519 98520 98521 98522 98523 98524 98525 98526 98527 98528
> > 98529 98530 98531 98532 98533 98534 98535 98536 98537 98538 98539 98540
> > 98541 98542 98543 98544 98545 98546 98547 98548 98549 98550 98551 98552
> > 98553 98554 98555 98556 98557 98558 98559 98560 98561 98562 98563 98564
> > 98565 98566 98567 98568 98569 98570 98571 98572 98573 98574 98575 98576
> > 98577 98578 98579 98580 98581 98582 98583 98584 98585 98586 98587 98588
> > 98589 98590 98591 98592 98593 98594 98595 98596 98597 98598 98599 98600
> > 98601 98602 98603 98604 98605 98606 98607 98608 98609 98610 98611 98612
> > 98613 98614 98615 98616 98617 98618 98619 98620 98621 98622 98623 98624
> > 98625 98626 98627 98628 98629 98630 98631 98632 98633 98634 98635 98636
> > 98637 98638 98639 98640 98641 98642 98643 98644 98645 98646 98647 98648
> > 98649 98650 98651 98652 98653 98654 98655 98656 98657 98658 98659 98660
> > 98661 98662 98663 98664 98665 98666 98667 98668 98669 98670 98671 98672
> > 98673 98674 98675 98676 98677 98678 98679 98680 98681 98682 98683 98684
> > 98685 98686 98687 98688 98689 98690 98691 98692 98693 98694 98695 98696
> > 98697 98698 98699 98700 98701 98702 98703 98704 98705 98706 98707 98708
> > 98709 98710 98711 98712 98713 98714 98715 98716 98717 98718 98719 98720
> > 98721 98722 98723 98724 98725 98726 98727 98728 98729 98730 98731 98732
> > 98733 98734 98735 98736 98737 98738 98739 98740 98741 98742 98743 98744
> > 98745 98746 98747 98748 98749 98750 98751 98752 98753 98754 98755 98756
> > 98757 98758 98759 98760 98761 98762 98763 98764 98765 98766 98767 98768
> > 98769 98770 98771 98772 98773 98774 98775 98776 98777 98778 98779 98780
> > 98781 98782 98783 98784 98785 98786 98787 98788 98789 98790 98791 98792
> > 98793 98794 98795 98796 98797 98798 98799 98800 98801 98802 98803 98804
> > 98805 98806 98807 98808 98809 98810 98811 98812 98813 98814 98815 98816
> > 98817 98818 98819 98820 98821 98822 98823 98824 98825 98826 98827 98828
> > 98829 98830 98831 98832 98833 98834 98835 98836 98837 98838 98839 98840
> > 98841 98842 98843 98844 98845 98846 98847 98848 98849 98850 98851 98852
> > 98853 98854 98855 98856 98857 98858 98859 98860 98861 98862 98863 98864
> > 98865 98866 98867 98868 98869 98870 98871 98872 98873 98874 98875 98876
> > 98877 98878 98879 98880 98881 98882 98883 98884 98885 98886 98887 98888
> > 98889 98890 98891 98892 98893 98894 98895 98896 98897 98898 98899 98900
> > 98901 98902 98903 98904 98905 98906 98907 98908 98909 98910 98911 98912
> > 98913 98914 98915 98916 98917 98918 98919 98920 98921 98922 98923 98924
> > 98925 98926 98927 98928 98929 98930 98931 98932 98933 98934 98935 98936
> > 98937 98938 98939 98940 98941 98942 98943 98944 98945 98946 98947 98948
> > 98949 98950 98951 98952 98953 98954 98955 98956 98957 98958 98959 98960
> > 98961 98962 98963 98964 98965 98966 98967 98968 98969 98970 98971 98972
> > 98973 98974 98975 98976 98977 98978 98979 98980 98981 98982 98983 98984
> > 98985 98986 98987 98988 98989 98990 98991 98992 98993 98994 98995 98996
> > 98997 98998 98999 99000 99001 99002 99003 99004 99005 99006 99007 99008
> > 99009 99010 99011 99012 99013 99014 99015 99016 99017 99018 99019 99020
> > 99021 99022 99023 99024 99025 99026 99027 99028 99029 99030 99031 99032
> > 99033 99034 99035 99036 99037 99038 99039 99040 99041 99042 99043 99044
> > 99045 99046 99047 99048 99049 99050 99051 99052 99053 99054 99055 99056
> > 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066 99067 99068
> > 99069 99070 99071 99072 99073 99074 99075 99076 99077 99078 99079 99080
> > 99081 99082 99083 99084 99085 99086 99087 99088 99089 99090 99091 99092
> > 99093 99094 99095 99096 99097 99098 99099 99100 99101 99102 99103 99104
> > 99105 99106 99107 99108 99109 99110 99111 99112 99113 99114 99115 99116
> > 99117 99118 99119 99120 99121 99122 99123 99124 99125 99126 99127 99128
> > 99129 99130 99131 99132 99133 99134 99135 99136 99137 99138 99139 99140
> > 99141 99142 99143 99144 99145 99146 99147 99148 99149 99150 99151 99152
> > 99153 99154 99155 99156 99157 99158 99159 99160 99161 99162 99163 99164
> > 99165 99166 99167 99168 99169 99170 99171 99172 99173 99174 99175 99176
> > 99177 99178 99179 99180 99181 99182 99183 99184 99185 99186 99187 99188
> > 99189 99190 99191 99192 99193 99194 99195 99196 99197 99198 99199 99200
> > 99201 99202 99203 99204 99205 99206 99207 99208 99209 99210 99211 99212
> > 99213 99214 99215 99216 99217 99218 99219 99220 99221 99222 99223 99224
> > 99225 99226 99227 99228 99229 99230 99231 99232 99233 99234 99235 99236
> > 99237 99238 99239 99240 99241 99242 99243 99244 99245 99246 99247 99248
> > 99249 99250 99251 99252 99253 99254 99255 99256 99257 99258 99259 99260
> > 99261 99262 99263 99264 99265 99266 99267 99268 99269 99270 99271 99272
> > 99273 99274 99275 99276 99277 99278 99279 99280 99281 99282 99283 99284
> > 99285 99286 99287 99288 99289 99290 99291 99292 99293 99294 99295 99296
> > 99297 99298 99299 99300 99301 99302 99303 99304 99305 99306 99307 99308
> > 99309 99310 99311 99312 99313 99314 99315 99316 99317 99318 99319 99320
> > 99321 99322 99323 99324 99325 99326 99327 99328
> > Multiply-claimed block(s) in inode 15: 98304 98305 98306 98307 98308 98309
> > 98310 98311 98312 98313 98314 98315 98316 98317 98318 98319 98320 98321
> > 98322 98323 98324 98325 98326 98327 98328 98329 98330 98331 98332 98333
> > 98334 98335 98336 98337 98338 98339 98340 98341 98342 98343 98344 98345
> > 98346 98347 98348 98349 98350 98351 98352 98353 98354 98355 98356 98357
> > 98358 98359 98360 98361 98362 98363 98364 98365 98366 98367 98368 98369
> > 98370 98371 98372 98373 98374 98375 98376 98377 98378 98379 98380 98381
> > 98382 98383 98384 98385 98386 98387 98388 98389 98390 98391 98392 98393
> > 98394 98395 98396 98397 98398 98399 98400 98401 98402 98403 98404 98405
> > 98406 98407 98408 98409 98410 98411 98412 98413 98414 98415 98416 98417
> > 98418 98419 98420 98421 98422 98423 98424 98425 98426 98427 98428 98429
> > 98430 98431 98432 98433 98434 98435 98436 98437 98438 98439 98440 98441
> > 98442 98443 98444 98445 98446 98447 98448 98449 98450 98451 98452 98453
> > 98454 98455 98456 98457 98458 98459 98460 98461 98462 98463 98464 98465
> > 98466 98467 98468 98469 98470 98471 98472 98473 98474 98475 98476 98477
> > 98478 98479 98480 98481 98482 98483 98484 98485 98486 98487 98488 98489
> > 98490 98491 98492 98493 98494 98495 98496 98497 98498 98499 98500 98501
> > 98502 98503 98504 98505 98506 98507 98508 98509 98510 98511 98512 98513
> > 98514 98515 98516 98517 98518 98519 98520 98521 98522 98523 98524 98525
> > 98526 98527 98528 98529 98530 98531 98532 98533 98534 98535 98536 98537
> > 98538 98539 98540 98541 98542 98543 98544 98545 98546 98547 98548 98549
> > 98550 98551 98552 98553 98554 98555 98556 98557 98558 98559 98560 98561
> > 98562 98563 98564 98565 98566 98567 98568 98569 98570 98571 98572 98573
> > 98574 98575 98576 98577 98578 98579 98580 98581 98582 98583 98584 98585
> > 98586 98587 98588 98589 98590 98591 98592 98593 98594 98595 98596 98597
> > 98598 98599 98600 98601 98602 98603 98604 98605 98606 98607 98608 98609
> > 98610 98611 98612 98613 98614 98615 98616 98617 98618 98619 98620 98621
> > 98622 98623 98624 98625 98626 98627 98628 98629 98630 98631 98632 98633
> > 98634 98635 98636 98637 98638 98639 98640 98641 98642 98643 98644 98645
> > 98646 98647 98648 98649 98650 98651 98652 98653 98654 98655 98656 98657
> > 98658 98659 98660 98661 98662 98663 98664 98665 98666 98667 98668 98669
> > 98670 98671 98672 98673 98674 98675 98676 98677 98678 98679 98680 98681
> > 98682 98683 98684 98685 98686 98687 98688 98689 98690 98691 98692 98693
> > 98694 98695 98696 98697 98698 98699 98700 98701 98702 98703 98704 98705
> > 98706 98707 98708 98709 98710 98711 98712 98713 98714 98715 98716 98717
> > 98718 98719 98720 98721 98722 98723 98724 98725 98726 98727 98728 98729
> > 98730 98731 98732 98733 98734 98735 98736 98737 98738 98739 98740 98741
> > 98742 98743 98744 98745 98746 98747 98748 98749 98750 98751 98752 98753
> > 98754 98755 98756 98757 98758 98759 98760 98761 98762 98763 98764 98765
> > 98766 98767 98768 98769 98770 98771 98772 98773 98774 98775 98776 98777
> > 98778 98779 98780 98781 98782 98783 98784 98785 98786 98787 98788 98789
> > 98790 98791 98792 98793 98794 98795 98796 98797 98798 98799 98800 98801
> > 98802 98803 98804 98805 98806 98807 98808 98809 98810 98811 98812 98813
> > 98814 98815 98816 98817 98818 98819 98820 98821 98822 98823 98824 98825
> > 98826 98827 98828 98829 98830 98831 98832 98833 98834 98835 98836 98837
> > 98838 98839 98840 98841 98842 98843 98844 98845 98846 98847 98848 98849
> > 98850 98851 98852 98853 98854 98855 98856 98857 98858 98859 98860 98861
> > 98862 98863 98864 98865 98866 98867 98868 98869 98870 98871 98872 98873
> > 98874 98875 98876 98877 98878 98879 98880 98881 98882 98883 98884 98885
> > 98886 98887 98888 98889 98890 98891 98892 98893 98894 98895 98896 98897
> > 98898 98899 98900 98901 98902 98903 98904 98905 98906 98907 98908 98909
> > 98910 98911 98912 98913 98914 98915 98916 98917 98918 98919 98920 98921
> > 98922 98923 98924 98925 98926 98927 98928 98929 98930 98931 98932 98933
> > 98934 98935 98936 98937 98938 98939 98940 98941 98942 98943 98944 98945
> > 98946 98947 98948 98949 98950 98951 98952 98953 98954 98955 98956 98957
> > 98958 98959 98960 98961 98962 98963 98964 98965 98966 98967 98968 98969
> > 98970 98971 98972 98973 98974 98975 98976 98977 98978 98979 98980 98981
> > 98982 98983 98984 98985 98986 98987 98988 98989 98990 98991 98992 98993
> > 98994 98995 98996 98997 98998 98999 99000 99001 99002 99003 99004 99005
> > 99006 99007 99008 99009 99010 99011 99012 99013 99014 99015 99016 99017
> > 99018 99019 99020 99021 99022 99023 99024 99025 99026 99027 99028 99029
> > 99030 99031 99032 99033 99034 99035 99036 99037 99038 99039 99040 99041
> > 99042 99043 99044 99045 99046 99047 99048 99049 99050 99051 99052 99053
> > 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065
> > 99066 99067 99068 99069 99070 99071 99072 99073 99074 99075 99076 99077
> > 99078 99079 99080 99081 99082 99083 99084 99085 99086 99087 99088 99089
> > 99090 99091 99092 99093 99094 99095 99096 99097 99098 99099 99100 99101
> > 99102 99103 99104 99105 99106 99107 99108 99109 99110 99111 99112 99113
> > 99114 99115 99116 99117 99118 99119 99120 99121 99122 99123 99124 99125
> > 99126 99127 99128 99129 99130 99131 99132 99133 99134 99135 99136 99137
> > 99138 99139 99140 99141 99142 99143 99144 99145 99146 99147 99148 99149
> > 99150 99151 99152 99153 99154 99155 99156 99157 99158 99159 99160 99161
> > 99162 99163 99164 99165 99166 99167 99168 99169 99170 99171 99172 99173
> > 99174 99175 99176 99177 99178 99179 99180 99181 99182 99183 99184 99185
> > 99186 99187 99188 99189 99190 99191 99192 99193 99194 99195 99196 99197
> > 99198 99199 99200 99201 99202 99203 99204 99205 99206 99207 99208 99209
> > 99210 99211 99212 99213 99214 99215 99216 99217 99218 99219 99220 99221
> > 99222 99223 99224 99225 99226 99227 99228 99229 99230 99231 99232 99233
> > 99234 99235 99236 99237 99238 99239 99240 99241 99242 99243 99244 99245
> > 99246 99247 99248 99249 99250 99251 99252 99253 99254 99255 99256 99257
> > 99258 99259 99260 99261 99262 99263 99264 99265 99266 99267 99268 99269
> > 99270 99271 99272 99273 99274 99275 99276 99277 99278 99279 99280 99281
> > 99282 99283 99284 99285 99286 99287 99288 99289 99290 99291 99292 99293
> > 99294 99295 99296 99297 99298 99299 99300 99301 99302 99303 99304 99305
> > 99306 99307 99308 99309 99310 99311 99312 99313 99314 99315 99316 99317
> > 99318 99319 99320 99321 99322 99323 99324 99325 99326 99327 99328
> > Pass 1C: Scanning directories for inodes with multiply-claimed blocks
> > Pass 1D: Reconciling multiply-claimed blocks
> > (There are 1 inodes containing multiply-claimed blocks.)
> >
> > File /test.img (inode #15, mod time Wed Dec 31 16:00:00 1969)
> >
> >   has 1025 multiply-claimed block(s), shared with 2 file(s):
> >         <filesystem metadata>
> >         <The group descriptor inode> (inode #7, mod time Mon Jul 23
> >         18:10:08
> >
> > 2018)
> > Clone multiply-claimed blocks<y>?
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > Pass 5: Checking group summary information
> >
> > /dev/sdg2: ***** FILE SYSTEM WAS MODIFIED *****
> >
> >           17 inodes used (0.00%, out of 1913184)
> >          
> >            0 non-contiguous files (0.0%)
> >            0 non-contiguous directories (0.0%)
> >            
> >              # of inodes with ind/dind/tind blocks: 2/2/0
> >              Extent depth histogram: 6
> >      
> >       283495 blocks used (3.71%, out of 7640576)
> >      
> >            0 bad blocks
> >            1 large file
> >            
> >            6 regular files
> >            2 directories
> >            0 character device files
> >            0 block device files
> >            0 fifos
> >            0 links
> >            0 symbolic links (0 fast symbolic links)
> >            0 sockets
> >
> > ------------
> >
> >            8 files
>
> I don't think mainline U-Boot supports octeon, but I might be wrong.
> Which U-Boot version is this ? Can you replicate this on ie. sandbox?
>
> --
> Best regards,
> Marek Vasut

While mainline U-Boot does not support Octeon, we have our own fork of it that
I maintain. I am using the 2018.07 release with only a few minor changes
around the periphery to support the older version of U-Boot Octeon is based
off of. I found one source of the corruption in ext4fs_get_new_inode_no()
where it is setting bgd->bg_itable_unused without cpu_to_le16, which causes
this to break on big-endian systems. Additionally, for 64-bit filesystems,
bg_itable_unused_high never gets set.

The biggest issue is the lack of cpu_to_le16() which causes incorrect data to
be written into the block group descriptor.

This is easy to see in the code and easy to replicate on a big-endian
platform.

I also saw some corruption occurring on our OcteonTX platform which is based
off of 2018.07 with minimal changes to U-Boot, though I need to investigate
this more.

Unrelated to this, one other thing to note is that I noticed that U-Boot does
not use extents when writing large files and is very slow. I think fixing this
would be a major project, though.

-Aaron

--
Aaron Williams
Senior Software Engineer
Cavium, Inc.
(408) 943-7198 (510) 789-8988 (cell)
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Cavium Octeon support for u-boot

Chris Packham
Hi Aaron,

On Wed, Jul 25, 2018 at 2:06 PM Aaron Williams
<[hidden email]> wrote:
> While mainline U-Boot does not support Octeon, we have our own fork of it that
> I maintain. I am using the 2018.07 release with only a few minor changes
> around the periphery to support the older version of U-Boot Octeon is based
> off of.

Just out of interest is there any particular barrier to Octeon support
landing upstream? At $dayjob we have 4 platforms using various Octeon
III processors. We're in the position of being stuck on an older
version of u-boot because we can't bring support for our boards up to
date without having to bring all the Octeon SoC support forward as
well. Is there any interest from Cavium to get Octeon upstreamed?
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: ext4: massive corruption with ext4write

Wolfgang Denk
In reply to this post by Aaron Williams
Dear Aaron,

In message <6444410.LK6WS1oxaM@awilliams-suse> you wrote:
>
...
> While mainline U-Boot does not support Octeon, we have our own fork of it that
> I maintain. I am using the 2018.07 release with only a few minor changes
> around the periphery to support the older version of U-Boot Octeon is based
> off of. I found one source of the corruption in ext4fs_get_new_inode_no()
> where it is setting bgd->bg_itable_unused without cpu_to_le16, which causes
> this to break on big-endian systems. Additionally, for 64-bit filesystems,
> bg_itable_unused_high never gets set.

So when you know the/one cause of the problems, and how to fix it,
then why don't you submit a patch?

Best regards,

Wolfgang Denk

--
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [hidden email]
Computers are not intelligent. They only think they are.
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: Cavium Octeon support for u-boot

Aaron Williams
In reply to this post by Chris Packham
The Octeon version of U-Boot is based off an older versin of U-Boot (2012.07)
with a lot of features backported. For example, I have backported NVME support
as well as the latest filesystem code. Since there have been significant
changes to U-Boot (i.e. 64-bit support) that were missing at the time it would
be a massive undertaking to support the current U-Boot with Octeon or to push
the Octeon support upstream. While in general the changes to the core U-Boot
were fairly minimal for Octeon, there were significant changes to board.c.

Hopefully we can get OcteonTX pushed upstream, however, since we are tracking
the current U-Boot releases. I'm not sure how to go about doing this, since I
would basically need a branch that I can maintain.

No significant new work is planned for the Octeon SDK or bootloader, however,
other than maintenance releases or adding features as needed by customers.

Although the Octeon code makes extensive use of the device tree, it does not
follow the dm model since that was not available when the U-Boot work took
place.

The way the Linux kernel and simple executive applications are started is also
unique to Octeon since multi-core support was also not present. This code is
complex due to the way it has evolved while trying to maintain backwards
compatibility.

Another potential barrier to upstreaming Octeon support is the fact that it
makes heavy use of our rather large SDK and the sheer amount of code. A quick
check with wc shows that we have around a million lines of code involved under
arch/mips/cpu/octeon, arch/mips/include/asm/arch-octeon and board/octeon/
common. This doesn't include the various drivers, either, or our custom
board.c file needed for the required flexibility. Around 434Klocs are the
hardware register definition files, which admittedly are huge due to the
autogenerated nature of them and all the comments and big/little endian
support. These files are shared by U-Boot, the Linux kernel and simple-
executive applications.

Some of the SDK and common board code is a mess since it tries to avoid
changes to the core of U-Boot. The networking and QLM configuration code is a
bit of a  mess since it has evolved over many years. I'm sorry to say that the
SFP/QSFP code I recently added just makes it worse due to the way it ties in
phy support. There's duplication of code between U-Boot and our SDK in the
networking code, especially for things like phy link support. Some phys also
just don't map cleanly into the U-Boot (or Linux for that matter) model. Inphi
(Cortina) and the Microsemi (Vitesse) in particular do not map into the one
phy per MAC model and all hell breaks loose with QSFPs where multiple MACs
share a phy or one MAC is split between multiple phys.

We also have implemented hooks in a number  of key places that are missing in
standard U-Boot. For example, we use a hook in the serial driver for a number
of boards to poll the SFP/QSFP ports to detect when a module is plugged in or
unplugged and to update activity LEDs.

We have also implemented something similar to native application support so
that simple-executive applications can make calls into U-Boot via a context
switching mechanism. This is used for filesystem access as well as some phy
operations with our 25G Liquid I/O boards.

We also run U-Boot in KSEG2 rather than KSEG0 so that U-Boot is always mapped
to the same virtual address regardless of where it is loaded in physical
memory. We do this for several reasons. First of all, it eliminates the need
to perform relocation fixups. Second of all, this frees up the lower 2GB of
memory space which can be a tight resource with a number of customers. While
we compile it as n32, it could conceivably be compiled as n64 but continue to
run in kseg2. n64 mode would simplify things by allowing the removal of using
cvmx_read_csr/cvmx_write_csr to instead use readq/writeq. The mapping between
virtual and physical addresses in our drivers would need to remain since there
is no IOMMU.

The code for bringing multiple cores out of reset and starting SE and the
Linux kernel would be tricky to port. There is a fair amount of assembly code
I've maintained on Octeon in order to do this.

Some of the fixes could also be pushed upstream, such as endian fixes we have
made in the USB and ext4 code.

Right now the main limitation to doing this is I'm pretty much the only one
maintaining Octeon support for U-Boot across all of our boards as well as
working on support for OcteonTX and OcteonTX2. A number of OcteonTX drivers
could easily be modified to work with Octeon, such as SPI, eMMC, NAND, i2c,
etc. The main difference in many cases is the fact that OcteonTX represents
devices as PCI devices (in addition to the device tree), whereas on Octeon
they are directly accessible and are only represented in the device tree.
There are huge differences, however, between OcteonTX and Octeon. With
OcteonTX, the memory and device tree are prepared by the BDK and ATF before U-
Boot is started. On Octeon, often U-Boot is the first thing that is loaded or
it is loaded by a simple SPI or eMMC bootloader and U-Boot is responsible for
memory initialization and a lot of other things.

-Aaron

On Wednesday, July 25, 2018 12:55:36 AM PDT Chris Packham wrote:

> External Email
>
> Hi Aaron,
>
> On Wed, Jul 25, 2018 at 2:06 PM Aaron Williams
>
> <[hidden email]> wrote:
> > While mainline U-Boot does not support Octeon, we have our own fork of it
> > that I maintain. I am using the 2018.07 release with only a few minor
> > changes around the periphery to support the older version of U-Boot
> > Octeon is based off of.
>
> Just out of interest is there any particular barrier to Octeon support
> landing upstream? At $dayjob we have 4 platforms using various Octeon
> III processors. We're in the position of being stuck on an older
> version of u-boot because we can't bring support for our boards up to
> date without having to bring all the Octeon SoC support forward as
> well. Is there any interest from Cavium to get Octeon upstreamed?

--
Aaron Williams
Senior Software Engineer
Cavium, Inc.
(408) 943-7198 (510) 789-8988 (cell)


_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: Cavium Octeon support for u-boot

Alexander Graf
Hi Aaron,

On 08/21/2018 08:07 AM, Aaron Williams wrote:

> The Octeon version of U-Boot is based off an older versin of U-Boot (2012.07)
> with a lot of features backported. For example, I have backported NVME support
> as well as the latest filesystem code. Since there have been significant
> changes to U-Boot (i.e. 64-bit support) that were missing at the time it would
> be a massive undertaking to support the current U-Boot with Octeon or to push
> the Octeon support upstream. While in general the changes to the core U-Boot
> were fairly minimal for Octeon, there were significant changes to board.c.
>
> Hopefully we can get OcteonTX pushed upstream, however, since we are tracking
> the current U-Boot releases. I'm not sure how to go about doing this, since I
> would basically need a branch that I can maintain.

What help exactly do you need here? We already do see partners shipping
with random downstream versions of U-Boot on Octeon TX and every time I
see them missing out on amazing efi_loader fixes, a tiny fraction of my
heart dies :).

So I would really like to see Octeon TX support simply developed fully
upstream. That way your partners (and again their downstreams) can base
on upstream U-Boot versions and are not tied to particular older
versions that may be lacking features or bug fixes.


Thanks,

Alex

>
> No significant new work is planned for the Octeon SDK or bootloader, however,
> other than maintenance releases or adding features as needed by customers.
>
> Although the Octeon code makes extensive use of the device tree, it does not
> follow the dm model since that was not available when the U-Boot work took
> place.
>
> The way the Linux kernel and simple executive applications are started is also
> unique to Octeon since multi-core support was also not present. This code is
> complex due to the way it has evolved while trying to maintain backwards
> compatibility.
>
> Another potential barrier to upstreaming Octeon support is the fact that it
> makes heavy use of our rather large SDK and the sheer amount of code. A quick
> check with wc shows that we have around a million lines of code involved under
> arch/mips/cpu/octeon, arch/mips/include/asm/arch-octeon and board/octeon/
> common. This doesn't include the various drivers, either, or our custom
> board.c file needed for the required flexibility. Around 434Klocs are the
> hardware register definition files, which admittedly are huge due to the
> autogenerated nature of them and all the comments and big/little endian
> support. These files are shared by U-Boot, the Linux kernel and simple-
> executive applications.
>
> Some of the SDK and common board code is a mess since it tries to avoid
> changes to the core of U-Boot. The networking and QLM configuration code is a
> bit of a  mess since it has evolved over many years. I'm sorry to say that the
> SFP/QSFP code I recently added just makes it worse due to the way it ties in
> phy support. There's duplication of code between U-Boot and our SDK in the
> networking code, especially for things like phy link support. Some phys also
> just don't map cleanly into the U-Boot (or Linux for that matter) model. Inphi
> (Cortina) and the Microsemi (Vitesse) in particular do not map into the one
> phy per MAC model and all hell breaks loose with QSFPs where multiple MACs
> share a phy or one MAC is split between multiple phys.
>
> We also have implemented hooks in a number  of key places that are missing in
> standard U-Boot. For example, we use a hook in the serial driver for a number
> of boards to poll the SFP/QSFP ports to detect when a module is plugged in or
> unplugged and to update activity LEDs.
>
> We have also implemented something similar to native application support so
> that simple-executive applications can make calls into U-Boot via a context
> switching mechanism. This is used for filesystem access as well as some phy
> operations with our 25G Liquid I/O boards.
>
> We also run U-Boot in KSEG2 rather than KSEG0 so that U-Boot is always mapped
> to the same virtual address regardless of where it is loaded in physical
> memory. We do this for several reasons. First of all, it eliminates the need
> to perform relocation fixups. Second of all, this frees up the lower 2GB of
> memory space which can be a tight resource with a number of customers. While
> we compile it as n32, it could conceivably be compiled as n64 but continue to
> run in kseg2. n64 mode would simplify things by allowing the removal of using
> cvmx_read_csr/cvmx_write_csr to instead use readq/writeq. The mapping between
> virtual and physical addresses in our drivers would need to remain since there
> is no IOMMU.
>
> The code for bringing multiple cores out of reset and starting SE and the
> Linux kernel would be tricky to port. There is a fair amount of assembly code
> I've maintained on Octeon in order to do this.
>
> Some of the fixes could also be pushed upstream, such as endian fixes we have
> made in the USB and ext4 code.
>
> Right now the main limitation to doing this is I'm pretty much the only one
> maintaining Octeon support for U-Boot across all of our boards as well as
> working on support for OcteonTX and OcteonTX2. A number of OcteonTX drivers
> could easily be modified to work with Octeon, such as SPI, eMMC, NAND, i2c,
> etc. The main difference in many cases is the fact that OcteonTX represents
> devices as PCI devices (in addition to the device tree), whereas on Octeon
> they are directly accessible and are only represented in the device tree.
> There are huge differences, however, between OcteonTX and Octeon. With
> OcteonTX, the memory and device tree are prepared by the BDK and ATF before U-
> Boot is started. On Octeon, often U-Boot is the first thing that is loaded or
> it is loaded by a simple SPI or eMMC bootloader and U-Boot is responsible for
> memory initialization and a lot of other things.
>
> -Aaron
>
> On Wednesday, July 25, 2018 12:55:36 AM PDT Chris Packham wrote:
>> External Email
>>
>> Hi Aaron,
>>
>> On Wed, Jul 25, 2018 at 2:06 PM Aaron Williams
>>
>> <[hidden email]> wrote:
>>> While mainline U-Boot does not support Octeon, we have our own fork of it
>>> that I maintain. I am using the 2018.07 release with only a few minor
>>> changes around the periphery to support the older version of U-Boot
>>> Octeon is based off of.
>> Just out of interest is there any particular barrier to Octeon support
>> landing upstream? At $dayjob we have 4 platforms using various Octeon
>> III processors. We're in the position of being stuck on an older
>> version of u-boot because we can't bring support for our boards up to
>> date without having to bring all the Octeon SoC support forward as
>> well. Is there any interest from Cavium to get Octeon upstreamed?


_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: Cavium Octeon support for u-boot

Suneel Garapati
On Tue, Aug 21, 2018 at 5:57 AM, Alexander Graf <[hidden email]> wrote:

> Hi Aaron,
>
> On 08/21/2018 08:07 AM, Aaron Williams wrote:
>>
>> The Octeon version of U-Boot is based off an older versin of U-Boot
>> (2012.07)
>> with a lot of features backported. For example, I have backported NVME
>> support
>> as well as the latest filesystem code. Since there have been significant
>> changes to U-Boot (i.e. 64-bit support) that were missing at the time it
>> would
>> be a massive undertaking to support the current U-Boot with Octeon or to
>> push
>> the Octeon support upstream. While in general the changes to the core
>> U-Boot
>> were fairly minimal for Octeon, there were significant changes to board.c.
>>
>> Hopefully we can get OcteonTX pushed upstream, however, since we are
>> tracking
>> the current U-Boot releases. I'm not sure how to go about doing this,
>> since I
>> would basically need a branch that I can maintain.
>
>
> What help exactly do you need here? We already do see partners shipping with
> random downstream versions of U-Boot on Octeon TX and every time I see them
> missing out on amazing efi_loader fixes, a tiny fraction of my heart dies
> :).
>
> So I would really like to see Octeon TX support simply developed fully
> upstream. That way your partners (and again their downstreams) can base on
> upstream U-Boot versions and are not tied to particular older versions that
> may be lacking features or bug fixes.

I maintain U-Boot for Octeon TX platforms at Cavium, which is based on
v2018.01 currently
and also tracking future releases.
efi loader support could be added but lacks customer use case as of now.
Upstream effort, atleast, for OcteonTX may kick off soon.

Regards,
Suneel

>
>
> Thanks,
>
> Alex
>
>
>>
>> No significant new work is planned for the Octeon SDK or bootloader,
>> however,
>> other than maintenance releases or adding features as needed by customers.
>>
>> Although the Octeon code makes extensive use of the device tree, it does
>> not
>> follow the dm model since that was not available when the U-Boot work took
>> place.
>>
>> The way the Linux kernel and simple executive applications are started is
>> also
>> unique to Octeon since multi-core support was also not present. This code
>> is
>> complex due to the way it has evolved while trying to maintain backwards
>> compatibility.
>>
>> Another potential barrier to upstreaming Octeon support is the fact that
>> it
>> makes heavy use of our rather large SDK and the sheer amount of code. A
>> quick
>> check with wc shows that we have around a million lines of code involved
>> under
>> arch/mips/cpu/octeon, arch/mips/include/asm/arch-octeon and board/octeon/
>> common. This doesn't include the various drivers, either, or our custom
>> board.c file needed for the required flexibility. Around 434Klocs are the
>> hardware register definition files, which admittedly are huge due to the
>> autogenerated nature of them and all the comments and big/little endian
>> support. These files are shared by U-Boot, the Linux kernel and simple-
>> executive applications.
>>
>> Some of the SDK and common board code is a mess since it tries to avoid
>> changes to the core of U-Boot. The networking and QLM configuration code
>> is a
>> bit of a  mess since it has evolved over many years. I'm sorry to say that
>> the
>> SFP/QSFP code I recently added just makes it worse due to the way it ties
>> in
>> phy support. There's duplication of code between U-Boot and our SDK in the
>> networking code, especially for things like phy link support. Some phys
>> also
>> just don't map cleanly into the U-Boot (or Linux for that matter) model.
>> Inphi
>> (Cortina) and the Microsemi (Vitesse) in particular do not map into the
>> one
>> phy per MAC model and all hell breaks loose with QSFPs where multiple MACs
>> share a phy or one MAC is split between multiple phys.
>>
>> We also have implemented hooks in a number  of key places that are missing
>> in
>> standard U-Boot. For example, we use a hook in the serial driver for a
>> number
>> of boards to poll the SFP/QSFP ports to detect when a module is plugged in
>> or
>> unplugged and to update activity LEDs.
>>
>> We have also implemented something similar to native application support
>> so
>> that simple-executive applications can make calls into U-Boot via a
>> context
>> switching mechanism. This is used for filesystem access as well as some
>> phy
>> operations with our 25G Liquid I/O boards.
>>
>> We also run U-Boot in KSEG2 rather than KSEG0 so that U-Boot is always
>> mapped
>> to the same virtual address regardless of where it is loaded in physical
>> memory. We do this for several reasons. First of all, it eliminates the
>> need
>> to perform relocation fixups. Second of all, this frees up the lower 2GB
>> of
>> memory space which can be a tight resource with a number of customers.
>> While
>> we compile it as n32, it could conceivably be compiled as n64 but continue
>> to
>> run in kseg2. n64 mode would simplify things by allowing the removal of
>> using
>> cvmx_read_csr/cvmx_write_csr to instead use readq/writeq. The mapping
>> between
>> virtual and physical addresses in our drivers would need to remain since
>> there
>> is no IOMMU.
>>
>> The code for bringing multiple cores out of reset and starting SE and the
>> Linux kernel would be tricky to port. There is a fair amount of assembly
>> code
>> I've maintained on Octeon in order to do this.
>>
>> Some of the fixes could also be pushed upstream, such as endian fixes we
>> have
>> made in the USB and ext4 code.
>>
>> Right now the main limitation to doing this is I'm pretty much the only
>> one
>> maintaining Octeon support for U-Boot across all of our boards as well as
>> working on support for OcteonTX and OcteonTX2. A number of OcteonTX
>> drivers
>> could easily be modified to work with Octeon, such as SPI, eMMC, NAND,
>> i2c,
>> etc. The main difference in many cases is the fact that OcteonTX
>> represents
>> devices as PCI devices (in addition to the device tree), whereas on Octeon
>> they are directly accessible and are only represented in the device tree.
>> There are huge differences, however, between OcteonTX and Octeon. With
>> OcteonTX, the memory and device tree are prepared by the BDK and ATF
>> before U-
>> Boot is started. On Octeon, often U-Boot is the first thing that is loaded
>> or
>> it is loaded by a simple SPI or eMMC bootloader and U-Boot is responsible
>> for
>> memory initialization and a lot of other things.
>>
>> -Aaron
>>
>> On Wednesday, July 25, 2018 12:55:36 AM PDT Chris Packham wrote:
>>>
>>> External Email
>>>
>>> Hi Aaron,
>>>
>>> On Wed, Jul 25, 2018 at 2:06 PM Aaron Williams
>>>
>>> <[hidden email]> wrote:
>>>>
>>>> While mainline U-Boot does not support Octeon, we have our own fork of
>>>> it
>>>> that I maintain. I am using the 2018.07 release with only a few minor
>>>> changes around the periphery to support the older version of U-Boot
>>>> Octeon is based off of.
>>>
>>> Just out of interest is there any particular barrier to Octeon support
>>> landing upstream? At $dayjob we have 4 platforms using various Octeon
>>> III processors. We're in the position of being stuck on an older
>>> version of u-boot because we can't bring support for our boards up to
>>> date without having to bring all the Octeon SoC support forward as
>>> well. Is there any interest from Cavium to get Octeon upstreamed?
>
>
>
> _______________________________________________
> U-Boot mailing list
> [hidden email]
> https://lists.denx.de/listinfo/u-boot
_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot
Reply | Threaded
Open this post in threaded view
|

Re: Cavium Octeon support for u-boot

Alexander Graf
On 08/22/2018 12:00 AM, Suneel Garapati wrote:

> On Tue, Aug 21, 2018 at 5:57 AM, Alexander Graf <[hidden email]> wrote:
>> Hi Aaron,
>>
>> On 08/21/2018 08:07 AM, Aaron Williams wrote:
>>> The Octeon version of U-Boot is based off an older versin of U-Boot
>>> (2012.07)
>>> with a lot of features backported. For example, I have backported NVME
>>> support
>>> as well as the latest filesystem code. Since there have been significant
>>> changes to U-Boot (i.e. 64-bit support) that were missing at the time it
>>> would
>>> be a massive undertaking to support the current U-Boot with Octeon or to
>>> push
>>> the Octeon support upstream. While in general the changes to the core
>>> U-Boot
>>> were fairly minimal for Octeon, there were significant changes to board.c.
>>>
>>> Hopefully we can get OcteonTX pushed upstream, however, since we are
>>> tracking
>>> the current U-Boot releases. I'm not sure how to go about doing this,
>>> since I
>>> would basically need a branch that I can maintain.
>>
>> What help exactly do you need here? We already do see partners shipping with
>> random downstream versions of U-Boot on Octeon TX and every time I see them
>> missing out on amazing efi_loader fixes, a tiny fraction of my heart dies
>> :).
>>
>> So I would really like to see Octeon TX support simply developed fully
>> upstream. That way your partners (and again their downstreams) can base on
>> upstream U-Boot versions and are not tied to particular older versions that
>> may be lacking features or bug fixes.
> I maintain U-Boot for Octeon TX platforms at Cavium, which is based on
> v2018.01 currently
> and also tracking future releases.
> efi loader support could be added but lacks customer use case as of now.

Just because the message didn't get to you doesn't mean no customer has
a use case for it :). In a lot of cases people do a list of options
first and then evaluate.

SLES for example will require either SBBR or EBBR compliance for target
platforms. Fair enough, we're a partner and not a customer. Given the
stressed notion security (and that means updates and timely fixes for
known vulnerabilities) is getting across the whole market thanks to
Spectre, I'm fairly confident you'll see more requests coming in sooner
rather than later. We already see heavily increased interest on SLES in
industrial embedded today.

> Upstream effort, atleast, for OcteonTX may kick off soon.

That's great to hear! I'll be happy to help out in whatever way I can
(and time permits).


Alex

_______________________________________________
U-Boot mailing list
[hidden email]
https://lists.denx.de/listinfo/u-boot