>> [missing,corrupt]=ABOVE_verify_cdf('/data/vlf/full_files/2016/*/*/**/*Full_Data*.dat') Starting ABOVE_verify_cdf. Found 2693543 files. missing = /data/vlf/cdf/l1/timeseries/2016/01/05/isll/19/abv_raw_isll_201601051922_v01.cdf /data/vlf/cdf/l1/timeseries/2016/01/07/barr/18/abv_raw_barr_201601071843_v01.cdf /data/vlf/cdf/l1/timeseries/2016/02/11/mcmu/01/abv_raw_mcmu_201602110144_v01.cdf /data/vlf/cdf/l1/timeseries/2016/05/19/mcmu/11/abv_raw_mcmu_201605191104_v01.cdf /data/vlf/cdf/l1/timeseries/2016/07/18/cmrs/06/abv_raw_cmrs_201607180648_v01.cdf /data/vlf/cdf/l1/timeseries/2016/08/13/cmrs/05/abv_raw_cmrs_201608130528_v01.cdf /data/vlf/cdf/l1/timeseries/2016/08/17/LUCK/05/abv_raw_LUCK_201608170507_v01.cdf /data/vlf/cdf/l1/timeseries/2016/12/09/cmrs/08/abv_raw_cmrs_201612090853_v01.cdf /data/vlf/cdf/l1/timeseries/2016/12/12/LUCK/21/abv_raw_LUCK_201612122119_v01.cdf corrupt = /data/vlf/cdf/l1/timeseries/2016/01/11/gill/05/abv_raw_gill_201601110526_v01.cdf ------------------ Typical failure resulting in missing file Processing /data/vlf/full_files/2016/07/18/cmrs/06/20160718_064800_cmrs_vlf-02_Full_Data.dat Deleting: /data/vlf/cdf/l1/timeseries/2016/07/18/cmrs/06/abv_raw_cmrs_201607180648_v01.cdf Creating CDF file: abv_raw_cmrs_201607180648_v01.cdf **WARNING: Failed to create CDF: /data/vlf/cdf/l1/timeseries/2016/07/18/cmrs/06/abv_raw_cmrs_201607180648_v01.cdf MException with properties: identifier: 'MATLAB:imagesci:cdflib:wrongNumberOfHyperPutVarDataElements' message: 'When specifying a dimension count of [ 2 ] and a record count of 1 with variable 'abv_raw_labl', exactly 48 elements must be provided.' cause: {0x1 cell} stack: [3x1 struct] Deleting CDF file. -------------- Contents of corrupt file: 00000000 7b 31 31 30 31 31 36 2c 30 35 32 36 30 30 2e 30 |{110116,052600.0| 00000010 30 30 2c 30 2c 2c 32 2e 31 2c 32 30 31 34 30 37 |00,0,,2.1,201407| 00000020 30 32 66 2c 67 69 6c 6c 2c 32 30 31 35 30 39 30 |02f,gill,2015090| 00000030 34 2c 76 6c 66 2d 31 30 2c 31 32 39 2c 38 31 2c |4,vlf-10,129,81,| 00000040 31 35 32 2c 30 2c 31 30 30 30 30 30 35 35 39 2c |152,0,100000559,| 00000050 33 47 2c 36 36 34 2c 38 30 30 30 30 2c 31 2c 32 |3G,664,80000,1,2| 00000060 32 37 32 38 34 32 30 7d f5 be cd 00 f5 1c cc f5 |2728420}........| Note the missing field on the second line. The corrupt CDF file was deleted for now. The data could probably be recovered if necessary.