Delaware Web Designers – Inclind, Inc Internet Professionals | Inclind, Inc – Delaware Web Designers – Professional Delaware Web Design Since 1999

Jan/10

5

Sync Files in Drupal Using Services and xmlrpc()

In a previous entry we explored content syncing/distributing using the Services module and XMLRPC in Drupal. We learned how to create a method, its callback, how to invoke it through xmlrpc from another server, and how to create or update a node based on that data.

Lets take a look at how you bring in files as part of that node as well. From the previous example, before calling node_save($node), I call another function to process files. Name it however you wish. My module is called homes_sync, so my hooks and functions begin with that as per Drupal standards.

1
2
3
homes_sync_get_files($node, $nid, $sessid);
// save the node object
node_save($node);

homes_sync_get_files() is passed the $node object and will return it after adding (or removing) data. To have a reliable file sync, we need to only update files if they are newer, and remove them if they no longer exist.

In this specific example, the project I am working on has two CCK file fields. One is for a floorplan (a PDF) and the other is an image field. In a node, there is only 1 floorplan, but can be up to 2 dozen images. If you had a more generic node that just has a CCK filefield of ‘client_files’ for example, your code would be slightly different.

Before we add any files to the node, the first thing we should do is compare the list of files from the remote site to the local site. We start that off by getting $files via xmlrpc, and loading the node object if $node->nid is present. If any are found on the local site, and not the remote site, we can safely remove them from the node object. The two arrays are iterated over and remove files if necessary.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
function homes_sync_get_files($node, $nid, $sessid) {
 
	$files = xmlrpc(REMOTE_SERVICES_URL, 'file.getHomeFiles', $sessid, $nid);
 
	// first, compare list of files from parent server with local files
	// if our local files has something the parent server doesnt have
	// it was probably deleted. so we need to unset it from the node object
	// when it gets passed back through node_save, cck should delete it proper
 
	if ($node->nid) {
 
		$current_files = node_load($node->nid);
 
		if ($current_files && !$files) {
			// we have files locally, but none on the parent server. 	
			// remove all
			unset($node->field_floorplan);
			unset($node->field_image);
			watchdog('homes_sync', 'Remote copy of @node no longer has files. Removing all local files for @node.', array('@node' => $node->title), WATCHDOG_NOTICE);
		} elseif ($current_files && $files) {
 
			// build comparison arrays
 
			foreach ($files as $remote_file) {
				$remote_files[] = $remote_file['filename'];	
			}
 
			foreach ($current_files->field_floorplan as $local_floorplan_file) {
				$local_files[] = $local_floorplan_file['filename'];
			}
 
			foreach ($current_files->field_image as $local_image_file) {
				$local_files[] = $local_image_file['filename'];
			}
 
			// now lets see whats missing
 
			if (is_array($local_files) && is_array($remote_files)) {
 
				foreach ($local_files as $local_file) {
					if (!in_array($local_file, $remote_files)) {
						// parent site does not have this file in the node anymore
						$files_to_delete[] = $local_file;
					}
				}
 
				if (!empty($files_to_delete) && is_array($files_to_delete)) {
					foreach ($files_to_delete as $key => $file_to_delete) {
						// CCK field arrays exist in node objects even if they have no data
						// we can assume if the first record is NULL, there is no data
 
						if ($node->field_floorplan[0] != NULL) {
							foreach ($node->field_floorplan as $file_key => $floorplan) {
								if ($file_to_delete == $floorplan['filename']) {
									watchdog('homes_sync', '@file file does not exist on parent server anymore for @node. Removing.', array('@file' => $floorplan['filename'], '@node' => $node->title), WATCHDOG_NOTICE);
									unset($node->field_floorplan[$file_key]);
								}
							}
						}
 
						if ($node->field_image[0] != NULL) {
							foreach ($node->field_image as $image_key => $image) {
								if ($file_to_delete == $image['filename']) {
									watchdog('homes_sync', '@image image does not exist on parent server anymore for @node. Removing.', array('@image' => $image['filename'], '@node' => $node->title), WATCHDOG_NOTICE);
									unset($node->field_image[$image_key]);
								}
							}
						}
						unset($floorplan);
						unset($image);
					}
				}
 
			}
		}
 
		unset($remote_files);
		unset($local_files);
	}

Now that we are through with our deletion process, we can begin processing the remote files. For each file id ($fid) we call xmlrpc and get all the data and the file itself and bring it down to our local site. Based on the file extension, we begin building a query so we can look to see if this file exists locally, and what its last timestamp was. We do this so we know if a file should be replaced or not. For example, if you have MyHome.PDF on your local site, and a new version is uploaded on the remote site, simply comparing file names is not good enough. With a filename and a timestamp comparison, you are certain that it should be processed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
	if ($files) {
 
		foreach ($files as $fid => $remote_file) {
			$file = xmlrpc(REMOTE_SERVICES_URL, 'file.get', $sessid, $fid);
 
			if ($file) {
				// set filepath to save to
				// decode our content file
 
				$file['filepath'] = file_directory_path().'/'.$file['filename'];
				$file_data = base64_decode($file['file']);
 
				// set content type db table, and field to search on.
				// Load up the CCK field
 
				$filetype = explode('.', $file['filename']);
 
				if ($filetype[1] == 'pdf') {
					$content_table = 'content_type_homes';
					$fid_field = 'field_floorplan_fid';
				} else {
					$content_table = 'content_field_image';
					$fid_field = 'field_image_fid';
				}
 
				$file_details = array();
 
				// check if it's the same file coming over
				// only applies if the node exists in the first place. otherwise its a new import on a new home
				// if there is a file match we dont want to do anything but skip the creation
				// i think if you dont keep the node->file field prefilled, and returns those fields empty, it causes the file deletion on the server
 
				if ($node->nid) {
 
					$result = db_query("SELECT f.filename, f.fid, f.filepath, f.timestamp FROM {files} f
											INNER JOIN {%s} ctype ON f.fid = ctype.%s
										WHERE f.filename = '%s'", $content_table, $fid_field, $file['filename']);
 
 
 
					$file_details = db_fetch_array($result);
 
				}
 
 
 
				if ($file_details && $file_details['timestamp'] < $file['timestamp']) {
 
					// if file exists and has not changed remotely, skip it
 
					watchdog('homes_sync', '@file already exists for @home. Skipping.', array('@file' => $file_details['filename'], '@home' => $node->title), WATCHDOG_NOTICE);
 
				} else { 
 
					// save file
					// REPLACE because we can be certain we will not have duplicate files
					// so we will only ever receive unique ones here
 
					file_save_data($file_data, $file['filepath'], FILE_EXISTS_REPLACE);
 
					//write record in files table
					drupal_write_record('files', $file);
 
					$result = db_query("SELECT fid,filepath FROM {files} WHERE filename = '%s'", $file['filename']);
					$file_details = db_fetch_object($result);
 
					$file_data = field_file_load($file_details->fid);
 
					watchdog('homes_sync', 'Uploaded @file into @home.', array('@file' => $full_file['filename'], '@home' => $node->title), WATCHDOG_NOTICE);
 
					if ($file_data) {
 
						if ($filetype[1] == 'pdf') {
							$node->field_floorplan[] = $file_data;
						} else {
							$node->field_image[] = $file_data;
						}
 
					}
 
				}
 
 
			}
		}
	}
 
	return $node;
}

After the file is saved to the file system and database, field_file_load() is called to load the file data. With a populated array now, we can add that to our node object by simply putting $node->field_floorplan[] = $file_data or $node->field_image[] = $file_data to append the existing array with data. The $node object is returned, and node_save($node) is immediately called, and our node data and file data are saved. Just like that, we have a simple content distribution system from one parent site to potentially many remote sites.

This is just a simple example, your mileage may vary. We are utilizing this technology to hook companies together under different Drupal installations on different servers. Using Services you could also distribute Views configurations, users, and system settings.

RSS Feed

No comments yet.

Leave a comment!

You must be logged in to post a comment.

<<

>>

Contact Us

Web: http://www.inclindinc.com
Phone: 302-856-2802
Email: info@inclind.com
Twitter: @inclindinc

About Us

Inclind Inc is a web development company on the east coast specializing in web design, web hosting, custom website design, website design, web applications, Coldfusion development, database design, MySQL / MSSQL database & consultation, PHP development, Wordpress themes, iPhone application development, Drupal hosting, Drupal development, Drupal module development, logo branding, business logic, custom application programming, Linux and Windows Server management and more.

Visit our website at inclind.com

Theme Design by devolux.org
delaware's premiere web design and development company. serving delaware, maryland, pennsylvania and beyond.
delaware web design - delaware website design - delaware web development - maryland web design - maryland website design - maryland web development - ocean city maryland web design - philadelphia web design - philadelphia web designers - washington dc web development - washington dc web design - custom drupal templates - drupal module development - custom drupal modules - custom drupal development - drupal ubercart support - drupal ecommerce