dmvpn

https://supportforums.cisco.com/document/115356/dmvpn-spoke-troubleshooting-order-and-component-responsibilities http://www.cisco.com/c/en/us/support/docs/security-vpn/dynamic-multi-point-vpn-dmvpn/116957-technote-dmvpn-00.html http://stor.balios.net/Live2011/BRKSEC-3012_10_a6-rev2.pdf

smartd

http://www.linuxjournal.com/content/know-when-your-drives-are-failing-smartd

# service smartd restart
Redirecting to /bin/systemctl restart  smartd.service
# WARNING: Your hard drive is failing
Device: /dev/sdc [USB Sunplus], 1 Currently unreadable (pending) sectors
WARNING: Your hard drive is failing
Device: /dev/sdc [USB Sunplus], 1 Offline uncorrectable sectors

# smartctl -A /dev/sdc
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-229.20.1.el7.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0003   239   190   021    Pre-fail  Always       -       3041
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       555
  5 Reallocated_Sector_Ct   0x0033   196   196   140    Pre-fail  Always       -       25
  7 Seek_Error_Rate         0x000f   200   200   051    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   038   038   000    Old_age   Always       -       45770
 10 Spin_Retry_Count        0x0013   100   100   051    Pre-fail  Always       -       0
 11 Calibration_Retry_Count 0x0012   100   100   051    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       550
190 Airflow_Temperature_Cel 0x0022   057   030   045    Old_age   Always   In_the_past 43
194 Temperature_Celsius     0x0022   107   080   000    Old_age   Always       -       43
196 Reallocated_Event_Count 0x0032   199   199   000    Old_age   Always       -       1
197 Current_Pending_Sector  0x0012   200   200   000    Old_age   Always       -       1
198 Offline_Uncorrectable   0x0010   200   200   000    Old_age   Offline      -       1
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0009   200   200   051    Pre-fail  Offline      -       0

[root@hp-server ~]# smartctl -l error /dev/sdc
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-229.20.1.el7.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
ATA Error Count: 8073 (device log contains only the most recent five errors)
 CR = Command Register [HEX]
 FR = Features Register [HEX]
 SC = Sector Count Register [HEX]
 SN = Sector Number Register [HEX]
 CL = Cylinder Low Register [HEX]
 CH = Cylinder High Register [HEX]
 DH = Device/Head Register [HEX]
 DC = Device Command Register [HEX]
 ER = Error register [HEX]
 ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 8073 occurred at disk power-on lifetime: 45761 hours (1906 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 ff 5f 65 74 40  Error: UNC at LBA = 0x0074655f = 7628127

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 40 08 5f 65 74 00 00      01:12:54.018  READ FPDMA QUEUED
  61 20 08 6f 38 f4 01 00      01:12:54.018  WRITE FPDMA QUEUED
  60 20 08 af 2e 73 00 00      01:12:54.006  READ FPDMA QUEUED
  61 80 08 ef 37 f4 01 00      01:12:54.005  WRITE FPDMA QUEUED
  60 80 08 57 6b 47 00 00      01:12:54.004  READ FPDMA QUEUED

Error 8072 occurred at disk power-on lifetime: 45761 hours (1906 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 ff d7 5e 86 40  Error: UNC at LBA = 0x00865ed7 = 8806103

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 80 08 d7 5e 86 00 00      01:12:43.715  READ FPDMA QUEUED
  61 68 08 3f 03 f4 01 00      01:12:43.714  WRITE FPDMA QUEUED
  60 80 08 df ab 88 00 00      01:12:43.713  READ FPDMA QUEUED
  61 80 08 bf 02 f4 01 00      01:12:43.712  WRITE FPDMA QUEUED
  60 80 08 5f ab 88 00 00      01:12:43.712  READ FPDMA QUEUED

Error 8071 occurred at disk power-on lifetime: 45761 hours (1906 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 ff 5f 9a 35 40  Error: UNC at LBA = 0x00359a5f = 3512927

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 80 08 5f 9a 35 00 00      01:12:00.714  READ FPDMA QUEUED
  61 08 08 7f bd 5e 00 00      01:12:00.713  WRITE FPDMA QUEUED
  61 08 08 77 bd 5e 00 00      01:12:00.713  WRITE FPDMA QUEUED
  61 80 08 87 1c f3 01 00      01:12:00.699  WRITE FPDMA QUEUED
  60 80 08 df 99 35 00 00      01:12:00.698  READ FPDMA QUEUED

Error 8070 occurred at disk power-on lifetime: 45761 hours (1906 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 ff 67 f6 32 40  Error: UNC at LBA = 0x0032f667 = 3339879

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 80 08 67 f6 32 00 00      01:08:00.328  READ FPDMA QUEUED
  61 80 08 47 74 f0 01 00      01:08:00.310  WRITE FPDMA QUEUED
  60 80 08 e7 f5 32 00 00      01:08:00.310  READ FPDMA QUEUED
  61 80 08 c7 73 f0 01 00      01:08:00.309  WRITE FPDMA QUEUED
  60 80 08 67 f5 32 00 00      01:08:00.308  READ FPDMA QUEUED

Error 8069 occurred at disk power-on lifetime: 45761 hours (1906 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 ff 67 7e 4e 40  Error: UNC at LBA = 0x004e7e67 = 5144167

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 80 08 67 7e 4e 00 00      01:07:35.101  READ FPDMA QUEUED
  61 80 08 3f 05 f0 01 00      01:07:35.000  WRITE FPDMA QUEUED
  60 80 08 e7 7d 4e 00 00      01:07:34.999  READ FPDMA QUEUED
  61 80 08 bf 04 f0 01 00      01:07:34.999  WRITE FPDMA QUEUED
  60 80 08 67 7d 4e 00 00      01:07:34.982  READ FPDMA QUEUED
Another one
/var/log/messages-20160228:Feb 22 07:44:57 guava smartd[770]: Device: /dev/sda [SAT], 24 Offline uncorrectable sectors
/var/log/messages-20160228:Feb 22 08:14:57 guava smartd[770]: Device: /dev/sda [SAT], 24 Currently unreadable (pending) sectors