<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Connecting the dots... &#187; synchronization</title>
	<atom:link href="http://blog.rajatpandit.com/tag/synchronization/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.rajatpandit.com</link>
	<description>Thoughts on Web Development, Infrastructure and Application Scalability</description>
	<lastBuildDate>Thu, 29 Dec 2011 13:21:11 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3</generator>
		<item>
		<title>More: Synchronization using phing</title>
		<link>http://blog.rajatpandit.com/2009/01/05/more-synchronization-using-phing/</link>
		<comments>http://blog.rajatpandit.com/2009/01/05/more-synchronization-using-phing/#comments</comments>
		<pubDate>Mon, 05 Jan 2009 20:37:59 +0000</pubDate>
		<dc:creator>rp</dc:creator>
				<category><![CDATA[Web Development]]></category>
		<category><![CDATA[Phing]]></category>
		<category><![CDATA[Symfony]]></category>
		<category><![CDATA[synchronization]]></category>

		<guid isPermaLink="false">http://blog.rajatpandit.com/?p=114</guid>
		<description><![CDATA[Reading more about automation stuff, I came across Phing which is like Apache Ant but uses php to write &#8220;tasks&#8221; classes. According the Phings website. PHing Is Not GNU make;...]]></description>
			<content:encoded><![CDATA[<p>Reading more about automation stuff, I came across <a href="http://phing.info">Phing</a> which is like Apache Ant but uses php to write &#8220;tasks&#8221;  classes. According the Phings website.</p>
<blockquote><p>
<strong>PH</strong>ing <strong>I</strong>s <strong>N</strong>ot <strong>G</strong>NU make; it&#8217;s a project build system based on Apache Ant. You can do anything with it that you could do with a traditional build system like GNU make, and its use of simple XML build files and extensible PHP &#8220;task&#8221; classes make it an easy-to-use and highly flexible build framework. Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.
</p></blockquote>
<p>So talking about synchronization, there is an interesting task called <b>&#8220;FileSyncTask&#8221;</b> for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer. FileSyncTask can copy or display directory contents and copy files, optionally using compression and recursion.</p>
<p>FileSyncTask can be used to synchronize Website trees from staging to production servers and to backup key areas of the filesystems, which is exactly what I was after. So here&#8217;s additional information on how to go about it. </p>
<p>There are 4 different ways of using FileSyncTask:</p>
<ul>
<li>For copying local files.</li>
<li>For copying from the local machine to a remote machine using a remote shell program as the transport (ssh).</li>
<li>For copying from a remote machine to the local machine using a remote shell program.</li>
<li>For listing files on a remote machine.</li>
</ul>
<p>The SSH client called by FileSyncTask uses settings from the build.properties file:<br />
[ini]<br />
sync.source.projectdir=/home/development.com/public<br />
sync.destination.projectdir=/home/staging.com<br />
sync.remote.host=server.com<br />
sync.remote.user=user<br />
sync.destination.backupdir=/home/staging.com/backup<br />
sync.exclude.file=/home/development.com/build/sync.exclude<br />
[/ini]</p>
<h2> Only listing files that have been modified</h2>
<pre class="brush: xml; title: ; notranslate">
&lt;taskdef name=&quot;sync&quot; classname=&quot;phing.tasks.ext.FileSyncTask&quot; /&gt;
&lt;sync
    sourcedir=&quot;${sync.source.projectdir}&quot;
    destinationdir=&quot;${sync.destination.projectdir}&quot;
    listonly=&quot;true&quot;
    verbose=&quot;true&quot; /&gt;
</pre>
<p>Excluding files that dont need to be synchronized (config files etc)</p>
<pre>
*~
.svn
public/images/uploads/*
build/*
log/*
tmp
</pre>
<p>Copying files to a remote machine</p>
<pre class="brush: xml; title: ; notranslate">
&lt;taskdef name=&quot;sync&quot; classname=&quot;phing.tasks.ext.FileSyncTask&quot; /&gt;
&lt;sync
    sourcedir=&quot;${sync.source.projectdir}&quot;
    destinationdir=&quot;${sync.remote.user}@${sync.remote.host}:${sync.destination.projectdir}&quot;
    excludefile=&quot;${sync.exclude.file}&quot;
    verbose=&quot;true&quot; /&gt;
</pre>
<p>So here&#8217;s the directory structure of the configuration files</p>
<pre>
development.com
|-- build
|   |-- build.properties
|   |-- build.xml
|   |-- sync.exclude
|   `-- sync.properties
`-- public
    `-- index.php
</pre>
<p>Finally the build file<br />
Phing runs using xml build files for directions on what tasks to run and other configuration parameters, here&#8217;s a sample one.</p>
<pre class="brush: xml; title: ; notranslate">
&lt;?xml version=&quot;1.0&quot; ?&gt;
&lt;project name=&quot;example&quot; basedir=&quot;.&quot; default=&quot;build&quot;&gt;
&lt;property name=&quot;version&quot; value=&quot;1.0&quot; /&gt;

&lt;!-- Public targets --&gt;
&lt;target name=&quot;sync:list&quot; description=&quot;List files&quot;&gt;
  &lt;phingcall target=&quot;-sync-execute-task&quot;&gt;
    &lt;property name=&quot;listonly&quot; value=&quot;true&quot; /&gt;
  &lt;/phingcall&gt;
&lt;/target&gt;

&lt;target name=&quot;sync&quot; description=&quot;Copy files&quot;&gt;
  &lt;phingcall target=&quot;-sync-execute-task&quot;&gt;
    &lt;property name=&quot;listonly&quot; value=&quot;false&quot; /&gt;
  &lt;/phingcall&gt;
&lt;/target&gt;

&lt;!-- Private targets --&gt;
&lt;target name=&quot;-init&quot; description=&quot;Load main settings&quot;&gt;
  &lt;tstamp /&gt;
  &lt;property file=&quot;build.properties&quot; /&gt;
&lt;/target&gt;

&lt;target name=&quot;-sync-execute-task&quot; depends=&quot;-init&quot;&gt;
  &lt;property file=&quot;sync.properties&quot; /&gt;
  &lt;if&gt;
    &lt;not&gt;
      &lt;isset property=&quot;sync.verbose&quot; /&gt;
    &lt;/not&gt;
    &lt;then&gt;
      &lt;property name=&quot;sync.verbose&quot; value=&quot;true&quot; override=&quot;true&quot; /&gt;
      &lt;echo message=&quot;The value of sync.verbose has been set to true&quot; /&gt;
    &lt;/then&gt;
  &lt;/if&gt;
  &lt;property name=&quot;sync.remote.auth&quot; value=&quot;${sync.remote.user}@${sync.remote.host}&quot; /&gt;
  &lt;taskdef name=&quot;sync&quot; classname=&quot;phing.tasks.ext.FileSyncTask&quot; /&gt;
  &lt;sync
    sourcedir=&quot;${sync.source.projectdir}&quot;
    destinationdir=&quot;${sync.remote.auth}:${sync.destination.projectdir}&quot;
    backupdir=&quot;${sync.remote.auth}:${sync.destination.backupdir}&quot;
    excludefile=&quot;${sync.exclude.file}&quot;
    listonly=&quot;${listonly}&quot;
    verbose=&quot;${sync.verbose}&quot; /&gt;
&lt;/target&gt;
&lt;/project&gt;
</pre>
<p>Now executing the task</p>
<pre class="brush: bash; title: ; notranslate">
$ phing sync:list
</pre>
<p>Which outputs</p>
<pre class="brush: bash; title: ; notranslate">
Buildfile: /home/development.com/build/build.xml
example &gt; sync:list:
[phingcall] Calling Buildfile '/home/development.com/build/build.xml'
            with target '-sync-execute-task'

example &gt; -init:
[property] Loading /home/development.com/build/build.properties

example &gt; -sync-execute-task:
[property] Loading /home/development.com/build/sync.properties
[echo] The value of sync.verbose has been set to true

Execute Command
----------------------------------------
rsync -razv --list-only -b --backup-dir

Sync files to remote server
----------------------------------------
Source:        /home/development.com/public
Destination:   user@server.com:/home/staging.com
Backup:        user@server.com:/home/staging.com/backup

Exclude patterns
----------------------------------------
*~
.svn
.htaccess
public/index.development.php
public/images/uploads/*
build/*
log/*
tmp/*

(list of files that have changed)

BUILD FINISHED
Total time: 1.9763 second
</pre>
<p>Excellent stuff, just the kind of sync script you would like per project, <a href="http://symfony-project.org">symfony</a> does a great job at synchronizing local and remote code as part of the <code>project:deploy</code> task and that has really spoilt me to look for similar ways for other projects.</p>
<p>Additional Reading:<br />
<a href="http://www.fedecarg.com/wiki/FileSyncTask">http://www.fedecarg.com/wiki/FileSyncTask</a><br />
<a href="http://phing.info/docs/guide/current/">http://phing.info/docs/guide/current/</a><br />
<a href="http://raphaelstolt.blogspot.com/2007/03/rolling-your-own-phing-task.html">Writing your own phing task</a></p>
<div id="in_post_ad_bottom_1" style="clear:both;margin:0;padding:0;"><div class="brp-bp-234">
<script type="text/javascript"><!--
google_ad_client = "ca-pub-4254382394977039";
/* brp-234x60-bp */
google_ad_slot = "7787511801";
google_ad_width = 234;
google_ad_height = 60;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div>
<div class="brp-bp-234">
<script type="text/javascript"><!--
google_ad_client = "ca-pub-4254382394977039";
/* brp-234x60-BP-1 */
google_ad_slot = "9111022353";
google_ad_width = 234;
google_ad_height = 60;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div></div><div style='clear:both'></div>]]></content:encoded>
			<wfw:commentRss>http://blog.rajatpandit.com/2009/01/05/more-synchronization-using-phing/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Synchronizing using rsync</title>
		<link>http://blog.rajatpandit.com/2009/01/05/synchronizing-using-rsync/</link>
		<comments>http://blog.rajatpandit.com/2009/01/05/synchronizing-using-rsync/#comments</comments>
		<pubDate>Mon, 05 Jan 2009 20:16:54 +0000</pubDate>
		<dc:creator>rp</dc:creator>
				<category><![CDATA[Web Development]]></category>
		<category><![CDATA[managing wordpress]]></category>
		<category><![CDATA[rsync]]></category>
		<category><![CDATA[synchronization]]></category>

		<guid isPermaLink="false">http://blog.rajatpandit.com/?p=112</guid>
		<description><![CDATA[So I run quite a few wordpress blogs now and time and again I like playing around with a plugin or a theme and want to ensure that I first...]]></description>
			<content:encoded><![CDATA[<p>So I run quite a few wordpress blogs now and time and again I like playing around with a plugin or a theme and want to ensure that I first try it out locally and then push it to the production version. Also is hard for me keep track of what files I have changed or modified and really wouldnt want to drop the entire folder on the ftp client.<br />
I had been thinking about a cleaner way to do it, one of the option was to push my files to SVN repo and then do a checkout on the production box. I should really be doing something like this now that I will be hacking my wordpress installations more than I have ever done but living dangerously right now, I decided to take the approach of syncing files using the good old rsync command. This post is really just to store the commands somewhere so that I can refer to it later. But in case you find it useful as well, then leave a comment.</p>
<p>So before I start, here are a couple of ways to get rsync locally depending on what you run as an operating system.</p>
<p>On Debian or Ubuntu Linux:</p>
<pre class="brush: bash; title: ; notranslate"># apt-get install rsync</pre>
<p>or</p>
<pre class="brush: bash; title: ; notranslate">$ sudo apt-get install rsync</pre>
<p>On RHEL (Red hat enterprise Linux (RHEL)</p>
<pre class="brush: bash; title: ; notranslate">#up2date rsync</pre>
<p>Or for yum based installation manager</p>
<pre class="brush: bash; title: ; notranslate">yum install rsync</pre>
<p>rsync command common options</p>
<ul>
<li>&#8211;delete : delete files that don&#8217;t exist on sender (system)</li>
<li>-v : Verbose (try -vv for more detailed information)</li>
<li>-e &#8220;ssh options&#8221; : specify the ssh as remote shell</li>
<li>-a : archive mode</li>
<li>-r : recurse into directories</li>
<li>-z : compress file data</li>
</ul>
<p>Synchronize a local directory with a remote directory</p>
<pre class="brush: bash; title: ; notranslate">$ rsync -r -a -v -e &quot;ssh -l luke&quot; --delete busybox.enterprise-starship.foo:/webroot/ /local/webroot</pre>
<p>Synchronize a remote directory with a local directory</p>
<pre class="brush: bash; title: ; notranslate">$ rsync -r -a -v -e &quot;ssh -l luke&quot; --delete /local/webroot o busybox.enterprise-starship.foo:/webroot</pre>
<p>Here&#8217;s the official documentation for rsync for more settings.<br />
<a href="http://samba.anu.edu.au/rsync/documentation.html">http://samba.anu.edu.au/rsync/documentation.html</a></p>
<div id="in_post_ad_bottom_1" style="clear:both;margin:0;padding:0;"><div class="brp-bp-234">
<script type="text/javascript"><!--
google_ad_client = "ca-pub-4254382394977039";
/* brp-234x60-bp */
google_ad_slot = "7787511801";
google_ad_width = 234;
google_ad_height = 60;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div>
<div class="brp-bp-234">
<script type="text/javascript"><!--
google_ad_client = "ca-pub-4254382394977039";
/* brp-234x60-BP-1 */
google_ad_slot = "9111022353";
google_ad_width = 234;
google_ad_height = 60;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div></div><div style='clear:both'></div>]]></content:encoded>
			<wfw:commentRss>http://blog.rajatpandit.com/2009/01/05/synchronizing-using-rsync/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>

