How to save multi-dimentional array values into CSV/Test file Python

Vishal Singh Source

I am using winpy 6.3. I have extracted a time series variable datasets (daily 1950-2004) using "xarray" based on my area of interest using lat/long bounds from a netcdf file.

code:

clt_subset = nc.variables['clt'][:,latli:latui , lonli:lonui]

print(clt_subset):
[[[  96.07967377   32.5813179    30.86773872 ...,   99.99996185
     99.99997711   99.99997711]
  [  93.75789642   86.78536987   46.51786423 ...,   99.99756622
     99.99769592   99.99931335]
  [  99.19438171   99.71717834   97.34263611 ...,   99.99707794
     99.99639893   99.93907928]
  ..., 
  [   7.65702724    1.1814307     4.02125835 ...,   39.58660126
     37.71473694   42.10451508]
  [   9.48283291   18.4249897    45.22411346 ...,   70.95629883
     72.82741547   72.89440155]
  [  33.2973175    46.50339508   88.39287567 ...,   98.50241089
     98.47457123   91.32685089]]

 [[  85.40306854   28.19069862   19.56433678 ...,   99.96898651
     99.99860382  100.        ]
  [  80.49911499   49.17562485   25.18140984 ...,   99.99198151
     99.99337006   99.99979401]
  [  99.9821167    91.44667816   78.83125305 ...,   99.99027252
     99.99280548   99.99995422]
  ..., 

so on..............

print (clt_subset.shape)
(20075, 22, 25)

Now I am not able to save this array into csv file with their time series values (rows) for each grid (lat/lon) combination using "datetime" function in different columns (22*25). The code is here:

# 2. Specify the exact time period you want:
start = datetime.datetime(1950,1,1,0,0,0)
stop = datetime.datetime(2004,12,1,0,0,0)

istart = netCDF4.date2index(start,time_var,select='nearest')
istop = netCDF4.date2index(stop,time_var,select='nearest')
print (istart,istop)

hs = clt_subset[istart:istop,latli:latui , lonli:lonui]
tim = dtime[istart:istop]

ts = pd.Series(hs,index=tim,name=clt_subset)
ts.to_csv('time_series_from_netcdf.csv')

while executing this, saying:

Error-

File "C:\python3\WinPython\python-3.6.5.amd64\lib\site-packages\pandas\core\series.py", line 3275, in _sanitize_array
    raise Exception('Data must be 1-dimensional')

Exception: Data must be 1-dimensional

When I am extracting values (20075) at only one location (single lat/lon), I am able to do this with this:

vname = 'clt'
#vname = 'surf_el'
var = nc.variables[vname]
hs = var[istart:istop,iy,ix]
tim = dtime[istart:istop]

# Create Pandas time series object
ts = pd.Series(hs,index=tim,name=vname)
#write to a CSV file
ts.to_csv('time_series_from_netcdf.csv')

I do not know where I am doing mistake?

pythonpython-3.xpython-2.7numpypython-xarray

Answers

answered 4 days ago Vishal Singh #1

I have solved this problem. The solution is here:

# suppose a 3D_array which contains (time series,latitude,longitude)
#for example: 3D_array.shape(1000,20,22)
#reshape your 3D array to 2D array for saving into csv or txt file

2Darray=np.resize(3D_array,[3D_array.shape[0],3D_array.shape[1]*3D_array.shape[2]])
np.savetxt("example.csv",2Darray,delimiter=',')

It will convert your 3D array to 2D array and then save to CSV file. Finally you will get time series values in rows and columns will represent grid locations (based on a lat/long). Thanks

comments powered by Disqus