Invalid symbolic link to directory with --parents

[murlock]

Tested with s5cmd 0.6.0

When using --parents parameter with cp, the symbolic link to a directory is managed as file

$ find tests -ls
  3318055      4 drwxrwxr-x   3 murlock  murlock      4096 Jun 17 11:59 test/
  3282596      0 lrwxrwxrwx   1 murlock  murlock        10 Jun 17 11:59 test/toto -> tata/160MB
  3282588      0 lrwxrwxrwx   1 murlock  murlock         4 Jun 17 11:59 test/tutu -> tata
  3281209      4 -rw-r--r--   1 murlock  murlock       111 Jun 17 11:58 test/magic
  3318056      4 drwxrwxr-x   2 murlock  murlock      4096 Jun 17 11:58 test/tata
  3281650      4 -rw-r--r--   1 murlock  murlock       111 Jun 17 11:58 test/tata/magic
  3282576 163844 -rw-rw-r--   1 murlock  murlock  167772160 Jun 17 11:58 test/tata/160MB
$ s5cmd  -endpoint-url http://127.0.0.1:5000 -us 5  cp --parents test/ s3://s5cmd/test2/
                    # Uploading magic... (111 bytes)
                    # Uploading 160MB... (167772160 bytes)
                    # Uploading magic... (111 bytes)
                    # Uploading toto... (167772160 bytes)
                    # Uploading tutu... (4096 bytes)
                    + "cp --parents test/magic s3://s5cmd/test2/magic"
                    + "cp --parents test/tata/magic s3://s5cmd/test2/tata/magic"
                    - "cp --parents test/tutu s3://s5cmd/test2/tutu": MultipartUpload: upload multipart failed upload id: ODgyNzQyZmUtNzkzYy00ZGNjLThjN2MtOGZkODRmMTdiOTJl caused by: EntityTooLarge: Your proposed upload exceeds the maximum allowed object size. status code: 400, request id: tx9cb2eb104ddb4fa0a26f2-005d079de3, host id: tx9cb2eb104ddb4fa0a26f2-005d079de3
                    + "cp --parents test/toto s3://s5cmd/test2/toto"
                    + "cp --parents test/tata/160MB s3://s5cmd/test2/tata/160MB"
2019/06/17 16:04:23 -ERR "cp test/ s3://s5cmd/test2/": Not all jobs completed successfully: 4/5
2019/06/17 16:04:24 # All workers idle, finishing up...

A strace show that symbolic link tutu was treated as file:

10057 newfstatat(AT_FDCWD, "test/tutu", {st_mode=S_IFDIR|0775, st_size=4096, ...}, 0) = 0                                           
10057 openat(AT_FDCWD, "test/tutu", O_RDONLY|O_CLOEXEC <unfinished ...>                                                             
10059 <... connect resumed> )           = -1 EINPROGRESS (Operation now in progress)                                                
10057 <... openat resumed> )            = 10      
10057 lseek(10, 0, SEEK_CUR)            = 0                                                                                         
10057 lseek(10, 0, SEEK_END)            = 9223372036854775807  
10061 pread64(10, 0xc00031c000, 32768, 1844674407370956) = -1 EISDIR (Is a directory)  

and network trace show the result of lseek used as-is:

    PUT /s5cmd/test2/tutu?partNumber=1&uploadId=NjEwZTE3MTYtMmUxNC00N2E4LWFjODgtOTFlMzYwYTM3NmJk HTTP/1.1\r\n
    Content-Length: 922337203685478\r\n

[igungor]

@murlock hey, that's a lot of debug information. Thanks!

s5cmd directory walking has no support for symbolic links unfortunately. This is on our roadmap. Once implemented, I think we add some --follow-symlink or similar parameter to support symbolic links to prevent breaking current behaviour.

[murlock]

Thanks !