I am not able to find a solution for delta upload to S3 bucket. Every time its uploading the entire folder.
Is there a provision for uploading only the changed + new files?
Adding EXTRA_ARGS: "--size-only" on the pipeline variable did the work for me..
As per document by AWS, `--size-only (boolean) Makes the size of each key the only criteria used to decide whether to sync from source to destination.`.
Hope this helps.
same issue here ... cant have partial sync whereas everything gets uploaded at any commit
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Online forums and learning are now in one easy-to-use experience.
By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.