5
Sync Files in Drupal Using Services and xmlrpc()
0 Comments | Posted by Kevin Quillen in Content Management, Custom Website Design, Drupal Development, How To's, PHP, Web 2.0, Web Applications, drupal modules
In a previous entry we explored content syncing/distributing using the Services module and XMLRPC in Drupal. We learned how to create a method, its callback, how to invoke it through xmlrpc from another server, and how to create or update a node based on that data.
Lets take a look at how you bring in files as part of that node as well. From the previous example, before calling node_save($node), I call another function to process files. Name it however you wish. My module is called homes_sync, so my hooks and functions begin with that as per Drupal standards.
1 2 3 | homes_sync_get_files($node, $nid, $sessid); // save the node object node_save($node); |
homes_sync_get_files() is passed the $node object and will return it after adding (or removing) data. To have a reliable file sync, we need to only update files if they are newer, and remove them if they no longer exist.
In this specific example, the project I am working on has two CCK file fields. One is for a floorplan (a PDF) and the other is an image field. In a node, there is only 1 floorplan, but can be up to 2 dozen images. If you had a more generic node that just has a CCK filefield of ‘client_files’ for example, your code would be slightly different.
Before we add any files to the node, the first thing we should do is compare the list of files from the remote site to the local site. We start that off by getting $files via xmlrpc, and loading the node object if $node->nid is present. If any are found on the local site, and not the remote site, we can safely remove them from the node object. The two arrays are iterated over and remove files if necessary.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | function homes_sync_get_files($node, $nid, $sessid) { $files = xmlrpc(REMOTE_SERVICES_URL, 'file.getHomeFiles', $sessid, $nid); // first, compare list of files from parent server with local files // if our local files has something the parent server doesnt have // it was probably deleted. so we need to unset it from the node object // when it gets passed back through node_save, cck should delete it proper if ($node->nid) { $current_files = node_load($node->nid); if ($current_files && !$files) { // we have files locally, but none on the parent server. // remove all unset($node->field_floorplan); unset($node->field_image); watchdog('homes_sync', 'Remote copy of @node no longer has files. Removing all local files for @node.', array('@node' => $node->title), WATCHDOG_NOTICE); } elseif ($current_files && $files) { // build comparison arrays foreach ($files as $remote_file) { $remote_files[] = $remote_file['filename']; } foreach ($current_files->field_floorplan as $local_floorplan_file) { $local_files[] = $local_floorplan_file['filename']; } foreach ($current_files->field_image as $local_image_file) { $local_files[] = $local_image_file['filename']; } // now lets see whats missing if (is_array($local_files) && is_array($remote_files)) { foreach ($local_files as $local_file) { if (!in_array($local_file, $remote_files)) { // parent site does not have this file in the node anymore $files_to_delete[] = $local_file; } } if (!empty($files_to_delete) && is_array($files_to_delete)) { foreach ($files_to_delete as $key => $file_to_delete) { // CCK field arrays exist in node objects even if they have no data // we can assume if the first record is NULL, there is no data if ($node->field_floorplan[0] != NULL) { foreach ($node->field_floorplan as $file_key => $floorplan) { if ($file_to_delete == $floorplan['filename']) { watchdog('homes_sync', '@file file does not exist on parent server anymore for @node. Removing.', array('@file' => $floorplan['filename'], '@node' => $node->title), WATCHDOG_NOTICE); unset($node->field_floorplan[$file_key]); } } } if ($node->field_image[0] != NULL) { foreach ($node->field_image as $image_key => $image) { if ($file_to_delete == $image['filename']) { watchdog('homes_sync', '@image image does not exist on parent server anymore for @node. Removing.', array('@image' => $image['filename'], '@node' => $node->title), WATCHDOG_NOTICE); unset($node->field_image[$image_key]); } } } unset($floorplan); unset($image); } } } } unset($remote_files); unset($local_files); } |
Now that we are through with our deletion process, we can begin processing the remote files. For each file id ($fid) we call xmlrpc and get all the data and the file itself and bring it down to our local site. Based on the file extension, we begin building a query so we can look to see if this file exists locally, and what its last timestamp was. We do this so we know if a file should be replaced or not. For example, if you have MyHome.PDF on your local site, and a new version is uploaded on the remote site, simply comparing file names is not good enough. With a filename and a timestamp comparison, you are certain that it should be processed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | if ($files) { foreach ($files as $fid => $remote_file) { $file = xmlrpc(REMOTE_SERVICES_URL, 'file.get', $sessid, $fid); if ($file) { // set filepath to save to // decode our content file $file['filepath'] = file_directory_path().'/'.$file['filename']; $file_data = base64_decode($file['file']); // set content type db table, and field to search on. // Load up the CCK field $filetype = explode('.', $file['filename']); if ($filetype[1] == 'pdf') { $content_table = 'content_type_homes'; $fid_field = 'field_floorplan_fid'; } else { $content_table = 'content_field_image'; $fid_field = 'field_image_fid'; } $file_details = array(); // check if it's the same file coming over // only applies if the node exists in the first place. otherwise its a new import on a new home // if there is a file match we dont want to do anything but skip the creation // i think if you dont keep the node->file field prefilled, and returns those fields empty, it causes the file deletion on the server if ($node->nid) { $result = db_query("SELECT f.filename, f.fid, f.filepath, f.timestamp FROM {files} f INNER JOIN {%s} ctype ON f.fid = ctype.%s WHERE f.filename = '%s'", $content_table, $fid_field, $file['filename']); $file_details = db_fetch_array($result); } if ($file_details && $file_details['timestamp'] < $file['timestamp']) { // if file exists and has not changed remotely, skip it watchdog('homes_sync', '@file already exists for @home. Skipping.', array('@file' => $file_details['filename'], '@home' => $node->title), WATCHDOG_NOTICE); } else { // save file // REPLACE because we can be certain we will not have duplicate files // so we will only ever receive unique ones here file_save_data($file_data, $file['filepath'], FILE_EXISTS_REPLACE); //write record in files table drupal_write_record('files', $file); $result = db_query("SELECT fid,filepath FROM {files} WHERE filename = '%s'", $file['filename']); $file_details = db_fetch_object($result); $file_data = field_file_load($file_details->fid); watchdog('homes_sync', 'Uploaded @file into @home.', array('@file' => $full_file['filename'], '@home' => $node->title), WATCHDOG_NOTICE); if ($file_data) { if ($filetype[1] == 'pdf') { $node->field_floorplan[] = $file_data; } else { $node->field_image[] = $file_data; } } } } } } return $node; } |
After the file is saved to the file system and database, field_file_load() is called to load the file data. With a populated array now, we can add that to our node object by simply putting $node->field_floorplan[] = $file_data or $node->field_image[] = $file_data to append the existing array with data. The $node object is returned, and node_save($node) is immediately called, and our node data and file data are saved. Just like that, we have a simple content distribution system from one parent site to potentially many remote sites.
This is just a simple example, your mileage may vary. We are utilizing this technology to hook companies together under different Drupal installations on different servers. Using Services you could also distribute Views configurations, users, and system settings.
No comments yet.
Leave a comment!
You must be logged in to post a comment.

