OPNsense Forum

English Forums => Hardware and Performance => Topic started by: roro on August 26, 2016, 01:02:47 pm

Title: [SOLVED] CAM status: ATA Status Error
Post by: roro on August 26, 2016, 01:02:47 pm
Hi Everyone,
I have 2 Soekris devices installed with CF-card running Opnsense 16.7.2-i386.

On both I get the following messages in the System Log File:
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): Error 5, Retries exhausted
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): RES: 51 04 6f 63 6b 45 45 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): CAM status: ATA Status Error
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): Retrying command
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): RES: 51 04 6f 63 6b 45 45 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): CAM status: ATA Status Error
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): Retrying command
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): RES: 51 04 6f 63 6b 45 45 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): CAM status: ATA Status Error
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): Retrying command
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): RES: 51 04 6f 63 6b 45 45 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): CAM status: ATA Status Error
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): Retrying command
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): RES: 51 04 6f 63 6b 45 45 00 00 01 00
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): CAM status: ATA Status Error
Aug 26 12:16:51    kernel: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00 01 00

It has to do with the disk, anyone seen this before and is there a solution for this?

Greets,
Rosie
Title: Re: CAM status: ATA Status Error
Post by: franco on August 26, 2016, 06:06:15 pm
Hi Rosie,

CF card and TRIM in the same post is suspicious.

Assuming this is a Nano image, can you edit your /etc/fstab accordingly:

DISCLAIMER: Editing the fstab should not be done lightly, system bootup may fail. That's why OPNsense code never touches the file beyond image building or the bsdinstaller's installation process.

Add " # notrim" (no quotes) at the very end of the root partition line (the device should be /dev/ufs/opnsense0) and reboot.

It'll disable TRIM and hopefully the errors go away as the CF card is likely unable to handle the command.


Cheers,
Franco
Title: Re: CAM status: ATA Status Error
Post by: roro on August 27, 2016, 01:24:42 am
Hi Franco,
thanks for the reply.
I have done research as well and found tunefs and fsck.
As I have a CF there are 2 slices.
ad0s1a has trim enabled and ad0s2a has trim disabled.
When I try to disabled it on ad0s1a i'm not able due to the fact that this slice is still readonly.
I thought by booting in second slice i could disable it, but still no success.
Can you tell me how to enable writing to the first slice.
Greetings,
Rosie
Title: Re: CAM status: ATA Status Error
Post by: franco on August 28, 2016, 02:02:29 pm
Didn't you mean it's still read-write? That's when you can't run the tunefs to modify.

If you use the fstab trick the RC system will do the disabling for you on reboot. That's why I mentioned it. :)


Cheers,
Franco
Title: Re: CAM status: ATA Status Error
Post by: roro on August 30, 2016, 07:31:14 pm
Hi Franco,
i'm still strugling with this one.
What I do:
First reboot and start Opnsense with option 2.
I do:
# cat /etc/fstab
/dev/ufs/OPNsense0 / ufs rw,async,noatime 1 1
# vi /etc/fstab
ex/vi: Error: /var/tmp/vi.recover/: Read-only file system
ex/vi: Modifications not recoverable if the session fails
ex/vi: Error: /etc/fstab: Read-only file system
ex/vi: Error: Unable to create temporary file: Read-only file system
#
Sorry, but I still don't know how to get this partition read write.
I'm missing knowledge to get this slice read write.
What is the trick?
Title: Re: CAM status: ATA Status Error
Post by: roro on August 30, 2016, 08:43:40 pm
Franco,
don't know what happened but system is back to OPNsense 15.7.18_1-i386 and i had to assing nics and ips again.
I was trying with serial connection and I reset the soekris system with reset button.
I will build the system up again and see what happens.
Keep you informed.
Roro
Title: Re: CAM status: ATA Status Error
Post by: roro on August 30, 2016, 08:58:23 pm
Well I booted system with OPNsense1, I will update this slice and see whats happening next.
Fetching the update files now. Upgrade in progress:
Fetching libevent2-2.0.22_1.txz: 100%  254 KiB 260.0kB/s    00:01
Fetching libedit-3.1.20150325_1.txz: 100%  119 KiB 121.8kB/s    00:01
Fetching ldns-1.6.17_5.txz: 100%  379 KiB 388.3kB/s    00:01
Fetching jansson-2.7_1.txz: 100%   39 KiB  40.4kB/s    00:01
Fetching idnkit-1.0_5.txz: 100%  184 KiB 188.5kB/s    00:01
Fetching gmp-5.1.3_2.txz: 100%  474 KiB 484.9kB/s    00:01
Fetching gettext-runtime-0.19.6.txz: 100%  144 KiB 147.7kB/s    00:01
Fetching freetype2-2.6.2.txz: 100%  535 KiB 547.4kB/s    00:01
Fetching easy-rsa-3.0.1.txz: 100%   31 KiB  32.1kB/s    00:01
Fetching dnsmasq-2.75_1,1.txz: 100%  257 KiB 262.7kB/s    00:01
Fetching dhcp6-20080615_5.txz: 100%  104 KiB 106.2kB/s    00:01
Fetching curl-7.46.0_2.txz: 100%    1 MiB   1.5MB/s    00:01
Fetching choparp-20150613.txz: 100%    7 KiB   7.2kB/s    00:01
Fetching ca_root_nss-3.21.txz: 100%  330 KiB 337.7kB/s    00:01
Fetching bind910-9.10.3P2.txz: 100%    6 MiB   5.9MB/s    00:01
Fetching apinger-0.6.1_4.txz: 100%   32 KiB  33.0kB/s    00:01
Fetching libucl-0.7.3_1.txz: 100%   79 KiB  80.7kB/s    00:01
Checking integrity... done (0 conflicting)
[1/74] Upgrading openssl from 1.0.2_4 to 1.0.2_6...
[1/74] Extracting openssl-1.0.2_6:  62%
Title: Re: CAM status: ATA Status Error
Post by: franco on August 30, 2016, 09:28:11 pm
I'm not exactly sure what you're trying to do. The second slice is dormant, so it wasn't up to date. It doesn't have a shared configuration directory, that's also by design. We'd either go ahead and make the second slice of nano usable, or we'd at some point see that the second slice isn't as useful. We've now seen the latter. :)

Boot from the first slice as usual (not single-user mode), edit /etc/fstab accordingly and reboot and TRIM will be off...


Cheers,
Franco
Title: Re: CAM status: ATA Status Error
Post by: roro on August 31, 2016, 10:57:24 am
The whole evening upgrading the opnsense cf i386 soekris box from version 15.x.x to 16.7.2 OpenSSL.
1  OPNsense
2  OPNsense

F6 PXE
Boot:  2

Took a long time but no errors at all.
As usuall I go from openssl to libressl and then the messages about:

(ada0:ata0:0:0:0): CAM status: ATA Status Error
etc.

appears again. After that I went to sleep.

Now I will go back to OpenSSL and see whats happening.

System is running now OPNsense 16.7.2 (i386/OpenSSL), but:

 0) Logout                             7) Ping host
 1) Assign Interfaces                  8) Shell
 2) Set interface(s) IP address        9) pfTop
 3) Reset the root password           10) Filter Logs
 4) Reset to factory defaults         11) Restart web interface
 5) Power off system                  12) Upgrade from console
 6) Reboot system                     13) Restore a configuration

Enter an option: (ada0:ata0:0:0:0): DSM TRIM. ACB: 06 01 00 00 00 40 00 00 00 00
 01 00
(ada0:ata0:0:0:0): CAM status: ATA Status Error
(ada0:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(ada0:ata0:0:0:0): RES: 51 04 20 3c 2f 74 74 00 00 01 00
(ada0:ata0:0:0:0): Retrying command

So I think the switch from OpenSSL to LibreSSL causes these messages in the system log.
Is this solvable in the installation or otherwise?

Greets.
Title: Re: CAM status: ATA Status Error
Post by: franco on August 31, 2016, 11:16:03 am
If it's not the TRIM support the SD card's first slice is wearing out. LibreSSL will not cause physical errors on your media. Note that your second slice is pretty young in terms of write cycles as opposed to the first one.
Title: Re: CAM status: ATA Status Error
Post by: roro on August 31, 2016, 03:08:00 pm
Recap:
after a few days of hard try and error.
Soekris net5501-70 serial.
Opnsense i386 cf-card.

Findings:
1. because I could not see much via serial (soekris 9600, opnsense 115200). fixed
2. the nanobsd boot menu. Here you can choose:
   1  OPNsense
   2  OPNsense

   F6 PXE
   Boot:  1

3. then the freebsd/opnsense logo will come, but it doesn't display well on serial output, so you miss the freebsd/opnsense menu with Multiuser, Singleuser etc.

4. then I had issues with VI and /etc/fstab. I choose to use EE to edit this file and I was able to add "# notrim".

5. Finally SUCCES. The trim messages are gone. Thanks to Franco's help.

root@opn01:~ # tunefs -p /dev/ad0s1a
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  512
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            1032
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)                                 OPNsense0

root@opn01:~ # tunefs -p /dev/ada0s1a
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  512
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            1032
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)                                 OPNsense0
root@opn01:~ #
root@opn01:~ #
Title: Re: CAM status: ATA Status Error
Post by: franco on August 31, 2016, 04:28:56 pm
Hi Rosie,

Sounds good. I'm not entirely sure why the system things the card is TRIM-capable but there were only ever two cases I heard of. I think chemlud had similar issues, we added # notrim upon his request. Maybe he has more background on this?

You can probably improve the console experience in terms of disabling the second console (new in 16.7.3) and changing the serial speed as well to match Soekris, all under System: Settings: Administration.


Cheers,
Franco
Title: Re: [SOLVED] CAM status: ATA Status Error
Post by: roro on September 03, 2016, 12:04:26 am
Hi Franco,
i'm still getting the messages again.
Is it also possible to turn of:
tunefs: soft updates: (-n)                                 enabled
Greets
Title: Re: [SOLVED] CAM status: ATA Status Error
Post by: roro on September 03, 2016, 12:07:29 am
and it driving crazy. >:(
Title: Re: [SOLVED] CAM status: ATA Status Error
Post by: franco on September 03, 2016, 10:18:54 am
Try " # notrim,nosoft" instead. :)
Title: Re: [SOLVED] CAM status: ATA Status Error
Post by: roro on September 03, 2016, 05:12:50 pm
Franco,
thanks for the support.
The messages are gone now.
Greetings.
Title: Re: [SOLVED] CAM status: ATA Status Error
Post by: hjint on January 09, 2018, 12:03:20 pm
Just feedback of similar trouble and what solved it.

I've setup two identical machines (same hardware like MB, HD, ram, etc) and had the same issues on one machine that the installation fail during the fdisk stage while the other machine installed without trouble. I've swopped harddisks and run the install on the other machine without a problem, but when swopping back the HD's the one machine gave boot errors. At the end of the day the source of the problem was the IDE cable, multiple generic IDE cables later non the wiser until I replaced it with a branded IDE cable and all trouble stopped there and then.