renate.utils.file module#
- renate.utils.file.get_aws_region()[source]#
Returns the name of the AWS region used during the execution.
- Return type:
str
- renate.utils.file.move_to_uri(src, dst, ignore_extensions=['.sagemaker-uploading', '.sagemaker-uploaded'])[source]#
Moves files in directory or file to directory or s3.
If the files exist they are overwritten. The files in the local directory are deleted.
- renate.utils.file.copy_to_uri(src, dst, ignore_extensions=['.sagemaker-uploading', '.sagemaker-uploaded'])[source]#
Copies files in directory or file to directory or s3.
If the files exist they are overwritten. The files in the local directory are preserved.
- renate.utils.file.maybe_download_from_s3(url, local_dir)[source]#
Tries to download a file from S3.
- Return type:
str
- renate.utils.file.download_folder_from_s3(src_bucket, src_object_name, dst_dir)[source]#
Downloads folder from S3 to local disk.
- Return type:
None
- renate.utils.file.upload_folder_to_s3(local_dir, s3_url=None, dst_bucket=None, prefix=None, ignore_extensions=['.sagemaker-uploading', '.sagemaker-uploaded'])[source]#
Uploads all files within a local folder to s3.
- Parameters:
- Return type:
None
- renate.utils.file.download_file_from_s3(src_bucket, src_object_name, dst)[source]#
Downloads file from S3 to local disk
- renate.utils.file.upload_file_to_s3(src, s3_url=None, dst_bucket=None, dst_object_name=None)[source]#
Upload a file to an S3 bucket
- renate.utils.file.extract_file(dataset_name, data_path, file_name)[source]#
Extract .zip or .tar depending on the flag files into folder named with dataset name.
- Return type:
None
- renate.utils.file.download_file(dataset_name, data_path, src_bucket, src_object_name, url, file_name)[source]#
A helper function to download data from URL or s3.
- Return type:
None
- renate.utils.file.download_and_unzip_file(dataset_name, data_path, src_bucket, src_object_name, url, file_name)[source]#
A helper function to download data .zips and uncompress them.
- Return type:
None