I'm possessing some images. When the final preprocess is done, final size of the image is nearly, 500,000 KB (0.5GB). there are 1500 such images. so total would cost 750GB. The size of my PC even is 450GB.
So I'm unable to store these data. Can I store then in an external hard drive? (1TB) and use to run for the Neural Network?
Will it workout?
PS: I tried running it in hard drive. But then I got the following error
Traceback (most recent call last):
File "C:\Users\hirplk\Desktop\unet\DSBmasking\dsb_masking.py", line 231, in <module>
scan = next(gen)
File "C:\Users\hirplk\Desktop\unet\DSBmasking\dsb_masking.py", line 61, in get_batches
scan = load_scan(INPUT_FOLDER + patient)
File "C:\Users\hirplk\Desktop\unet\DSBmasking\dsb_masking.py", line 26, in load_scan
slices = [dicom.read_file(path + '/' + s) for s in os.listdir(path)]
File "C:\Users\hirplk\Desktop\unet\DSBmasking\dsb_masking.py", line 26, in <listcomp>
slices = [dicom.read_file(path + '/' + s) for s in os.listdir(path)]
File "C:\Research\Python_installation\lib\site-packages\dicom\filereader.py", line 589, in read_file
fp = open(fp, 'rb')
PermissionError: [Errno 13] Permission denied: 'D:/DSB/stage1/00edff4f51a893d80dae2d42a7f45ad1/00cba091fa4ad62cc3200a657aeb957e'
This code is what I'm trying to run
enter link description herehttps://github.com/rdcolema/dsb/blob/master/unet_node_masking.ipynb
In the line,
save_array("prepd_samples/{}.npy".format(patient), masked)
The saved image is nearly 5000,000 KB. How much of size can I reduce when I compress it? Will it be a performance hit? When running them in NN will it get too late to run?
The files are in external hard drive,
D:/DSB/stage1/00edff4f51a893d80dae2d42a7f45ad1/00cba091fa4ad62cc3200a657aeb957e
I right clicked on it. but no tab for security, to set the permission.
The permission error is generating at the place
slices = [dicom.read_file(path + '/' + s) for s in os.listdir(path)]