FolderShare REST Release Notes

New features:

REST "get-configuration" response includes user information.

Like most REST requests, the "get-configuration" request now returns the user's ID, account name, and display name.

Bugs fixed:

FALSE, zero, and empty value fields are improperly returned.

Fixed. When the REST client requests the values for file or folder fields that contain zero, FALSE, or empty strings, the field is not returned. Instead it now returns the field regardless of the value.

New features:

REST client host, port, user name, and password can be read from a JSON file.

Instead of using command-line options on the REST client, the name of the host, its port, the user's login name, and their password, all can be stored in a JSON file named using the "--auth" option or stored in an automatically searched ".foldershareAuth" file in the current folder or in the user's home folder. The file can contain multiple host and user names, selected by the "--host" option on the REST client.

REST client now supports a "json" output format.

The "--format json" command-line option selects command output in JSON, which can be easily parsed by most scripting languages.

REST client "upload" and "download" support "--delete" to delete extra content.

When synchronizing local and remote folders, if the destnation side has extra files or folders not present in the source, the "--delete" option directs that they be deleted. This keeps source and destination identical.

REST client "upload" and "download" support "--dry-run" to show what would be affected, but not do it.

When synchronizing local and remote folders, the "--dry-run" option shows a list of all files that would be uploaded or downloaded, what folders would be created, and what files or folders would be deleted, but no action is taken.

REST client "upload" command performance improved.

The upload command now issues far fewer server requests during an upload, improving performance.

Bugs fixed:

REST client requires a host name even just to show command help.

Fixed. When no host name is given, options to get help for a command complain that there is no host. Instead the client will now show any help information without requiring a host.

REST client password prompt shows the password.

Fixed. When no password is given on the command line and the REST client prompts for it, the password's typed letters are visible. Instead it now disables typed letters for Linux, BSD, macOS, and other POSIX OSes. The letters are still visible in Windows due to limitations in the OS.

REST client help claims it will prompt for the user name, but it doesn't.

Fixed. The help information says that if a "--username" option is not given, then the command will prompt for a user name. Instead it correctly tries to connect anonymously. The help information has been corrected.

REST client commands that output booleans in "text" format, output an empty string instead.

Fixed. Output formatting did not properly map boolean true/false values to "true" and "false" output strings. It does now.

REST client "hostname" command doesn't support "keyvalue" or "full" formats.

Fixed. The command's output did not support "keyvalue" or "full" formats, or the new "json" format. Now it does.

REST client "ls" command doesn't recurse with "-R" unless "linux" format is used.

Fixed. The "-R" option requests a recursive folder listing that only worked with the default "linux" output. It now works with "keyvalue", "full", and the new "json" output formats.

REST client "ls" command outputs "full" format when "keyvalue" requested.

Fixed. When the "keyvalue" output format was selected, the command output a PHP object "full" format dump instead. It now outputs key-value pairs.

REST client "ls" command with multiple operands doesn't distinguish between each operand's listing.

Fixed. When an "ls" command is given two or more operands (e.g. "ls /a /b") the output for each operand's listing was concatenated without indicating which output went with which operand. The output is now structured depending upon the output format and each listing is prefixed with the operand being listed.

REST client "ls" command's "keyvalue" format outputs as a PHP dump, not as keyword:value.

Fixed. The "ls" command incorrectly treated the "keyvalue" format in the same way as the "full" format, which outputs a PHP object dump. Instead it is now treated the same as the "text" format, which outputs a list of keyword:value lines for the files and folders being listed.

REST client "stat" command with multiple operands doesn't distinguish between each operand's listing.

Fixed. When an "stat" command is given two or more operands (e.g. "stat /a /b") the output for each operand's listing was concatenated without indicating which output went with which operand. The output is now structured depending upon the output format and each listing is prefixed with the operand being listed.

REST client "update" command generates server error on bad field name.

Fixed. When trying to set the value for a field for a file or folder, a typo in the field name creates an invalid name that causes a PHP error on the server. Instead, field names are now validated in the REST client.

REST client "update" command cannot take multiple arguments, despite help.

Fixed. The help information for the "update" command says multiple files and folders may be updated at the same time, but trying to do so gets an error message. The command now works as advertised.

Many REST client commands do not support different output formats.

Fixed. Commands that do not normally output anything were ignoring the output format choice and falling back on a "full" format if they did output. The commands now output different formats properly. Commands affected include "chown", "cp", "mkdir", "mv", "rm", "rmdir", "update", "version", and "whoami".

REST client "upload" and "download" commands accept "-n" and "--sync" options together, but shouldn't.

Fixed. The "-n" option requests transfers that do not overwrite existing files, and instead rename transferred files with a unique sequence number suffix. The "--sync" option requests that transfers only occur if files are newer. But "--sync" cannot work if it cannot see a file with the same name, which makes the combination of "-n" and "--sync" unworkable. The two options now cannot be used together.

REST client "xattr" command does not output anyting except when using the "full" format.

Fixed. When using the "keyvalue" or "linux" formats, the command outputs nothing. Instead it now supports these formats.

REST client "xattr" command doesn't sort output by keyword.

Fixed. The command outputs a list of keyword-value pairs for file or folder attributes, but the list was in an undefined order. It is now sorted.

REST client "xattr" claims to show the "systemhidden" attribute, but can't.

Fixed. The "systemhidden" attribute is an internal file and folder attribute used to mark items that are in the process of being deleted. Such items therefore do not really exist any more and any attempt to get their attributes correctly returns a "Not found" error. Since items with this attribute set are never found, the "systemhidden" attribute can only be shown for non-hidden items where it is always "false". This makes it pointless to show so the feature has been removed.

REST client "upload" command doesn't handle symbolic links properly.

Fixed. When uploading a folder that contains a symbolic link to another file or folder, the name and path to the link's target, rather of the link, was used to create and name remote folders. Instead it now uses the name and path to the link.

New features:

Date/time stamps on files are always preserved on upload and download.

Previously, the -p option was required to preserve date/time stamps.

The upload and download -u option is now a synonym for --sync.

Previously, the --sync option was a synonym for -pu. But with -p no longer needed, --sync and -u are now equivalent.

A "/" is no longer required at the start of all remote paths.

Remote paths without a leading "/" automatically refer to the user's home folder.

Upload and download progress messages show more detail on file sizes.

Previously file sizes were converted to KB, MB, GB, or TB and truncated to a whole number. Now a single digit after the decimal is included.

Bugs fixed:

Upload and download are confused by file/folder names using special characters, such as *, +, (, or ).

Fixed. File/folder names were inadvertantly passed to a function that treated special characters as part of a regular expression.

Upload sync always uploads image files, even when they haven't changed, but it properly skips non-image files that haven't changed.

Fixed. Uploaded images were not properly detected on sync and were therefore re-uploaded.

Upload sync uploads into FOLDER the first time, then into FOLDER/FOLDER for all further runs.

Fixed. Upload did not correctly detect and re-use a named remote FOLDER during non-first upload syncs.

Removed features:

The upload and download -p option is deprecated.

Previously, the -p option was required to preserve date/time stamps. This is now the default behavior and the option is no longer needed.

New features:

On module install, the module checks that PHP ZIP support is installed.

The module may use ZIP to uncompress downloaded multi-file content.

On module install, the module checks that FolderShare 1.3 is installed.

The module requires the 8.x-1.3 version of FolderShare due to API changes.

Bugs fixed:

Functions deprecated in Drupal 8.8 should not be called.

Fixed.

New features:

REST resource API calls to FolderShare updated to use new methods.

The FolderShare module renamed some file-handling APIs to use more consistent and descriptive naming. This module has been updated to use the new APIs.

"no overwrite" forms of file upload now supported.

The "new-rootfile-no-overwrite" and "new-file-no-overwrite" REST protocol messages create new files without overwriting existing files. The new files have a unique sequence number attached (e.g. "myfile-1.txt", "myfile-2.txt").

Last modified date passed on file upload and folder create.

The file and folder upload REST protocol messages now support a "Last-modified" HTTP header that contains the last modified date to use for the newly created file or folder. This may be used to keep local and remote content in sync by tracking modification dates.

Bugs fixed:

Domain names and port numbers are not being validated.

Invalid domain names and port numbers could be entered, such as by entering "http://example.com" instead of "example.com". Because the domain name was not validated, this erroneous syntax floated down into the underlying CURL where it cause changes in the way CURL issued requests. Domain names and port numbers are now validated and erroneous syntax generates an error.

Download to a new file fails with a creation error.

Local file path parsing and testing was incorrect and failed when a download was supposed to create an explicitly named destination file. This has been fixed.

Download file paths handled oddly.

Remote and local file path parsing was not robust. This has been improved.

Error messages not written to STDERR.

Error messages are now written to STDERR.

Paths are not being normalized on the client.

File paths that contain multiple adjacent "/"s or a trailing "/" are not normalized. While they can be handled on the server, normalizing them (and validating their structure) on the client allows for a faster turn-around when the user has entered something incorrectly.

Upload file paths handled oddly.

Remote and local file path parsing was not robust. This has been improved.

User names and passwords not being validated.

User names and passwords have a required length and structure for Drupal. Rather than send malformed user names and passwords to the server and await its error response, these are now validated in the client to have the right structure. Only if correctly formatted are they sent to the server, which could still reject them for being an unknown user name or bad password.

Changes:

FolderShareConnect::download() now has a $flags argument to control downloading.

The method downloads the indicated file or folder. The optional $flags argument is a bitmask that selects special download behaviors. Those behaviors control overwriting of existing local files, setting downloaded files and folders to use the remote item's timestamp, and updating local files only if remote files are newer. See the API documentation for more details.

FolderShareConnect::download() now downloads folders one descendant at a time.

Previously, downloading a remote folder caused that folder to be ZIPped and then downloaded. This could take a long time or fail when remote files and folders were large and it prevented the new update behaviorthat downloads files only if they have changed. The new download gets files and folders as needed, recursing through remote folders. The downloaded content is not ZIPped.

FolderShareConnect::download() and FolderShareConnect::upload() rewritten to have more robust local and remote file path handling.

Upload and download follow the similar semantics of a Linux/BSD/macOS cp command, so they support a variety of local and remote file path forms. This has been rewritten to improve its robustness and the variety of forms allowed.

FolderShareConnect::getFileOrFolder() now has a $recurse argument to recurse through folders.

The original method returned information about a single remote file or folder. This behavior remains when the new $recurse argument is FALSE. When TRUE, the method recurses downwards through folders and returns the indicated item and all its descendants in an array.

FolderShareConnect::upload() now has a $flags argument to control downloading.

The method uploads the indicated file or folder. The optional $flags argument is a bitmask that selects special upload behaviors. Those behaviors control overwriting of existing remote files, setting uploaded files and folders to use the local item's timestamp, and updating remote files only if local files are newer. See the API documentation for more details.

Client API packaged into a ZIP file.

The client API files are included in the module distribution. Previously they were individual PHP files in a "src" directory. This was confusing because the files are not needed except when building custom clients. These files are now packaged into a ZIP file.

New features:

All Linux/BSD/macOS command-like functionality moved to the new FolderShareCommands class

The functions to implement each of the client's Linux/BSD/macOS-like commands have been moved out of main's global name space and into the new FolderShareCommands class as public static methods.

All arguments for remote paths with "path" in the name have been renamed to use "uri".

Remote file or folder paths are URIs that may include a scheme ("personal" or "private") and user name or ID prefix. To reflect this, all arguments that refer to remote paths and used "path" in the argument name have been renamed with "uri" as the name. The functionality remains the same.

REST client splitPath() moved to FolderShareConnect::parseUri()

The function to parse remote file or folder URIs has been renamed and moved out of main's global name space and into the FolderShareConnect class as a public static method.

REST client normalizePath() and normalizePaths() moved to FolderShareConnect::normalizeUri() and FolderShareConnect:normalizeUris()

The functions to normalize remote file or folder URIs by removing redundant slashes have moved out of main's global name space and into the FolderShareConnect class as a public static methods.

Bugs fixed:

ls -t should sort by modification time with most recent first, but it has the reverse order.

Sorting has been fixed.

ls -R (recursive list) skips some folders.

Recursion accidentally overwrote an internal list of what subfolders to continue onwards and list. This truncated the recursion and skipped some folders.

Changes:

sync command renamed to wait.

The command waits for remote copies or moves to complete, so it has been renamed.

New features:

chown (change owner) command added.

For administrator users, the command changes the ownership of a file or folder.

download -n added.

The "-n" option prevents a download from overwriting an existing local file. Instead it will automatically add a unique sequence number. For example, if file "A.txt" is downloaded but there is already a local file with that name, the download will go into "A-2.txt".

download -p added.

The "-p" option preserves timestamps by setting the modification and access time of downloaded files and folders to match the modification time on the remote item. Without the option, downloaded files and folders have the current date as the modification time. This optional has the same affect as on the Linux/BSD/macOS cp command.

download -u added.

The "-u" option updates local files only if the remote file is newer or if the local file or folder does not exist. This should be used together with the "-p" option to insure that local files have comparable timestamps. This optional has the same affect as on the Linux/BSD/macOS cp command.

download -v shows download progress.

Previously, the "-v" option only printed the name of each file as it was downloaded. While this behavior remains, the size of the downloaded item and the number of bytes and percentage downloaded so far is included and updated during the download to show progress.

download --sync added.

The "--sync" option is a synonym for "-p" and "-u" together.

ls -1 added.

The "-1" option forces the file/folder list output to be in a single column. This is useful when the output must be parsed by a script. This optional has the same affect as on the Linux/BSD/macOS ls command.

ls -f added.

The "-f" option causes listed items to be output unsorted. This optional has the same affect as on the Linux/BSD/macOS ls command.

ls -h added.

The "-h" option causes shows sizes with KB/MB/GB suffixes. This optional has the same affect as on the Linux/BSD/macOS ls command.

ls -r added.

The "-r" option reverses the sort order. This optional has the same affect as on the Linux/BSD/macOS ls command.

mkdir -p added.

The "-p" causes all missing directories on a file path to be created automatically. For example "mkdir -p /a/b/c" will create "a" if it isn't there, then "b", then "c". This optional has the same affect as on the Linux/BSD/macOS mkdir command.

upload -n added.

The "-n" option prevents a upload from overwriting an existing remote file. Instead it will automatically add a unique sequence number. For example, if file "A.txt" is uploaded but there is already a remote file with that name, the upload will go into "A-2.txt".

upload -p added.

The "-p" option preserves timestamps by setting the modification time of uploaded files and folders to match the modification time on the local item. Without the option, uploaded files and folders have the current date as the modification time. This optional has the same affect as on the Linux/BSD/macOS cp command.

upload -u added.

The "-u" option updates remote files only if the local file is newer or if the remote file or folder does not exist. This should be used together with the "-p" option to insure that remote files have comparable timestamps. This optional has the same affect as on the Linux/BSD/macOS cp command.

upload -v shows upload progress.

Previously, the "-v" option only printed the name of each file as it was uploaded. While this behavior remains, the size of the uploaded item and the number of bytes and percentage uploaded so far is included and updated during the upload to show progress.

upload --sync added.

The "--sync" option is a synonym for "-p" and "-u" together.

Bugs fixed:

Move or copy of item into its own parent deletes it.

Fixed. Moving an item into its own parent is not a valid move or copy and it now reports an error.

New features:

REST web services moved to new "FolderShare REST" module.

Optional REST web services have been moved out of the FolderShare module and into this support module. This enables REST services to be updated on a schedule separate from that of the primary module.

REST web services command-line client moved into module.

The separately-distributed command-line client "foldershare" has been moved into the module release in the "client" directory. See the module's help page for examples of its use.

Module has a dedicated help page.

The module has its own REST-focused help page.

Removed features:

REST plugin reset to defaults button removed.

While REST features were in the FolderShare module, a "Restore" button on the administrator settings page would reset the REST plugin configuration to defaults. With the move to a new REST-focused module, there is no longer an administrator settings page. The REST plugin configuration is also so straight-forward that a "Restore" button seems superflous. Default REST plugin settings are discussed in the module's help page.

New features:

Client "ls -l" column width for permission too narrow.

Permission text is one of "public", "private", "personal", "shared with you", or "shared by you". The column width was 7 characters, and is now dynamic to accomodate the widest permission text.

Client "ls -l" column width for account name too narrow.

Account name text can be as little as one character, or as long as 60 characters. The column width was 8 characters, and is now dynamic to accomodate the widest account name.

Bugs fixed:

Client refers to undefined CURLE_REMOTE_ACCESS_DENIED.

The name is defined by libcurl, but not by the PHP CuRL package. Use of the name has been removed.

Client fails if server module does not have the same module version number.

A new protocol version number has been added that is separate from the module version number. The client checks that the server's protocol version is supported during client-server handshake.

Client does not print an account name in "ls -l" if an item is owned by anonymous.

Drupal's anonymous account is special and it actually has no account name. So, technically, printing an empty account name is correct. But not useful. The client now prints the anonymous user's display name (usually "Anonymous") when the account name is empty.

Client prints some error messages with HTML tags.

Error messages created on the server sometimes include HTML tags because the error messages are intended for display by a browser. For REST, these same messages are returned to the client. The client now strips HTML tags from error messages before printing them.