Fixing stuck backups/clearing the backup queue

Fixing stuck backups/clearing the backup queue

Previous backups may be queued up or have gone stale, in which case the queue will need to be flushed manually to resolve.

Windows based systems

  1. Stop the online backup scheduler service
  2. Start task manager, go to processes and kill/end any processes that are “Backup engine agent” AND “Backup scheduler” listed in this article (need to show processes as all users)
  3. Clear out the “ipc” folder under the program files location
  4. Clear out the “ipc” folder under the backup profile location (eg: .acb\ipc OR .obm\ipc)
  5. Start the online backup scheduler service again
  6. See if backups run as per normal
Posted in File backup: Desktop: issues with backup, File backup: Server: issues with backup | Tagged , , | Leave a comment

A nonrecoverable I/O error occurred on file

Error

[Microsoft][ODBC SQL Server Driver][SQL Server]A nonrecoverable I/O error occurred on file “C:\cybersecure-spool\1314935863830\XXXX.bak.ADD:” 112(failed to retrieve text for this error. Reason: 15100).

Cause

This can be caused by a stale spooling location.

Resolution

Clear the spooling location for the backup set and try the backup again.

Posted in MS-SQL Errors | Leave a comment

Unexpected exception…. Data error (cyclic redundancy check)

Error

[Error][New File] File=”XXXX.tmp” Error=”[com.ahsay.obc.core.bset.file.A] [BackupRequestTask.uploadFiles] Unexpected exception for sFilename=\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy52\XXXX sUploadPath=I:\XXXX.tmp sFileType=F, caused by [java.io.IOException] [ReadAheadInputStream.Reader.run]@829610816 Throwable=Data error (cyclic redundancy check). , caused by [java.io.IOException] Data error (cyclic redundancy check). “

Cause

This is due to corruption on the filesystem, commonly caused by failing hardware – most often by a disk with a bad block or failed RAID array that has had corruption propagated through to the data.

Resolution

  1. Follow our guide on checking the event log for hardware issues to confirm that the underlying hardware does not have errors.
  2. Once the underlying hardware is confirmed to be healthy, run a chkdsk or similar filesystem integrity check.
  3. Identify the files from the error and try to read them independently to determine their integrity – deleting or repairing these files may be necessary.
Posted in File backup: Desktop: general issues, File backup: Server: general issues | Leave a comment

ShadowCopy\ipc\running (Access is denied)

Error

[BackupIpcThread.doRunException] error: java.io.FileNotFoundException: C:\ProgramData\CyberSecure Advanced\ShadowCopy\ipc\running (Access is denied) Message: C:\ProgramData\CyberSecure Advanced\ShadowCopy\ipc\running (Access is denied)

Cause

The backup cannot access the ipc\running file via VSS, typically caused by multiple backup sets being run simultaneously on the same volume, which creates a conflict.

Resolution

  1. Stagger the backups to be at least 30 minutes apart.
  2. OR Switch off VSS in the backup set.
Posted in File backup: Server: general issues | Leave a comment

SQL error “Fail to backup Database”

Error

Fail to backup Database

Cause

Stale spooling location.

Resolution

Flush the spooling location.

Posted in MS-SQL Errors | Leave a comment

How to check the state of VSS writers in Windows

  1. Open a cmd.exe session as an Administrator-level user
  2. run the following command: vssadmin list writers
  3. check the output for your desired VSS writer, here’s an example of a healthy Exchange VSS writer:

    Writer name: ‘Microsoft Exchange Writer’
    Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
    Writer Instance Id: {3d54c40b-2bd3-4240-807a-2d02e344f706}
    State: [1] Stable
    Last error: No error

Posted in Uncategorized | Leave a comment

[Thread][ERS][BackupReport][BackupReport.send] Error sending backup job report for User=XXXX

Error

[Thread][ERS][BackupReport][BackupReport.send] Error sending backup job report for User=’swc’ BackupSet=’1381879818962′ BackupJob=’2013-11-18-21-00-00′

Unable to send email Server=smtp1.cybersecure.com.au: Invalid Addresses

Cause

  • This is caused by at least one of the backup contacts having an invalid email address.
  • Can also be caused by the email address containing semicolons, commas or spaces.

Resolution

  • Correct the email address(es) listed in the backup contacts, this can be done 2x ways:
  • To send to multiple email addresses, create a separate backup contact for each email address – do not use semicolons or other separating characters.
Posted in File backup: Desktop: general issues, File backup: Server: general issues | Leave a comment

Failed to GetUserProfileDirectory for wcDomain=’XXXX’ username=’XXXX’. (2) The system cannot find the file specified. ‘.

Error

Error running backup set Reason = ‘[WindowsUtil:createProcessAsUserN] Failed to GetUserProfileDirectory for wcDomain=’XXXX’ username=’XXXX’.  (2) The system cannot find the file specified.’.

Cause

The local user defined in the “User Authentication for Windows” section does not have a local profile setup.

Resolution

Update the “User Authentication for Windows” section with the current local user profile (check the home.txt file under the CyberSecure program installation path).

Posted in File backup: Server: general issues | Leave a comment

The writer failed due to an error that would likely not occur if the entire backup, restore, or shadow copy creation process was restarted. (VSS_E_WRITERERROR_RETRYABLE)

Error

The writer failed due to an error that would likely not occur if the entire backup, restore, or shadow copy creation process was restarted. (VSS_E_WRITERERROR_RETRYABLE)

Cause

This is most often caused by the Microsoft Exchange information store VSS writer being in an unstable state.

Resolution

  1. Check the exchange backup schedule is set to the recommended settings
  2. Check the event viewer for any hardware, exchange or resource (lack of RAM or paging file) issues.
  3. Restart the Microsoft Exchange Information Store service.
  4. Run “vssadmin list writers” – each writer should now report “State: stable” and “Last error: no error”
  5. Try the backup again

Note: the exchange VSS writer going regularly into an unstable state could mean underlying hardware, driver or resource issues.

We have seen this issue on machines that are memory overloaded and/or have very slow disk configurations, to properly resolve the issue: a) more RAM should be added to the machine and b) faster disk configuration should be upgraded on the machine.

Further reading

  • (Microsoft KB article): Time-out errors occur in Volume Shadow Copy service writers, and shadow copies are lost during backup and during times when there are high levels of input/output. Applies to Windows 2003, however the concepts are similar – VSS requires complex interprocess calls which can fail from a component that doesn’t respond in time.
Posted in Exchange Errors | Leave a comment

The writer operation failed because of a time-out between the Freeze and Thaw event. (VSS_E_WRITERERROR_TIMEOUT)

Error

[com.ahsay.afc.microsoft.MSExchangeExpt] The writer operation failed because of a time-out between the Freeze and Thaw event.  (VSS_E_WRITERERROR_TIMEOUT)

Cause

The exchange VSS writer has gone into a inconsistent state. If this issue occurs regularly it may indicate an underlying hardware issue or lack of resources (ie. system RAM).

The output of vssadmin list writers may look similar to:

Writer name: ‘Microsoft Exchange Writer’
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {088e9271-a67d-4bd4-82c6-b4b20c59fff4}
State: [9] Failed
Last error: Timed out

Resolution

  1. Check to determine whether the Exchange VSS writer state is showing as “stable”. If the writer is not stable, check the event log for hardware/disk events
  2. Restart the Microsoft Information Store service
  3. Run vssadmin list writers again and confirm that the exchange VSS writer is now stable
Posted in Exchange Errors | Leave a comment