Placing several backups in a safe place and keeping files synchronized either locally or remotely is an urgent necessity for all users whose everyday tasks solely rely on the use of computers. Hard drives get damaged, files get deleted, memory sticks get lost …etc. You have to make synchronized backups of your stuff!
Many backup applications are there. Some are GUI-based, while others run through the command line. Personally, I prefer to use the command line due to its high flexibility. One of the most widely used command for synchronization purposes has been the rsync command.
By the way, one of the reasons behind my migration from Windows to Linux was the lack of the FREE synchronization tools that run under Windows. Anyway, if I start talking about this again I won’t finish, and I already wrote about this in an earlier post. So, let’s start talking about “rsync” command.
rsync is an open source utility that provides fast incremental file transfer. “rsync” is freely available under the GNU General Public License
Features of rsync are already mentioned in different sites, but this is how I see it:
- Incremental File Transfer
- Compression while Transferring
- Synchronizing Remote Machines
- Encrypted and Secure
- Better Alternative to FTP
- Very Flexible
If you have a look at the rsync manual you will notice a huge number of arguments, and that might discourage you from using the command. Well, don’t worry! All those arguments are for advanced usages, and as the synchronization process gets more complex you will need to understand more arguments and when to use them; otherwise, you only need to used to little arguments to do you the job. Therefore , you will find yourself running the same command over and over with same arguments all the time.
Here is the Generic Syntax:
rsync [options] Src Dst
Where Src and Dst respectively are:
Src: Source location from where files will be synchronized. Dst: Destination location to where files will be synchronized.
Src and Dst can hold these location combinations except for Remote to Remote directories:
Local: "/directory" Remote: "username@hostname:"
Note: Remote Directory will be the HOME directory of the username used at the Remote machine by default!
A full synchronization process implies what is in this table will take place:
A very frequently used arguments with “rsync” command:
rsync -avz -u --progress --delete Src Dst -n
|-z||Compress while transferring files. This will make the transferring faster and will save the network bandwidth|
|-u||Update files which are newer in the Src|
|--progress||Show the transfer progress of each file while transferring|
|--delete||Delete files from Dst if they don’t exist in Src|
|-n||Preview mode. Changes won’t take effect till this argument is omitted|
Sync, without Sub-Directories
One of the syncing tasks I perform on a daily basis is that I sometimes want to sync a certain directory without syncing the subdirectories. This is how I do it:
rsync -dvz -u --progress --delete Src Dst -n
If you want the same effect that the argument ‘-a’ have, such as preserving links, owner, group …etc. Then, this is another option:
rsync -dlptgoDvz -u --progress --delete Src Dst -n
Arguments Worthy Looking at
|--exclude=PATTERN||exclude files matching PATTERN|
|--include=PATTERN||don’t exclude files matching PATTERN|
|--existing||skip creating new files on receiver|
|--ignore-existing||skip updating files that exist on receiver|
- Synchronization process always works from Src to Dst. So, Any change in the Src will be reflected in the Dst.
- A probable confusing matter here is the deleted files. Just remember, a deleted file in the Src will be deleted from the Dst, but any file deleted from the Dst will be recreated from the Src while syncing!
- As a rule of thumb, always make the necessary modification in the Src then sync it with the Dst.
- Be careful with the Trailing Slash. If the Trailing Slash appears, then content of that directory will be synced, otherwise, that directory AND its content will be synced. Try it out and see what happens!
Too Many Arguments!!
If you found any difficulty in using the proper arguments of rsync command due to their numbers, then I suggest you start with GUI rsync Application called Grsync. The good thing about it is that it shows you the arguments that are going to be used to perform the synchronization process!
What are SVN and Trac?
SVN: is a free/open-source Version Control System. Subversion (SVN) manages files and directories over time. Files are placed into a central repository and every change every made to the files or directories are remembered.
Trac: is an enhanced wiki and issue tracking system for software development projects. It provides a Graphical front end to SVN where diffs in files can be obtained. It cannot update the SVN repository. It simply provides a Project Management interface, wiki, ticketing system, and SVN front end.
sudo apt-get install apache2 sudo apt-get install subversion sudo apt-get install libapache2-svn sudo apt-get install trac sudo apt-get install libapache2-mod-fastcgi libapache2-mod-fcgid
Setting Up SVN and Trac
To make things simple and flexible in case you want to make some changes in the future, configurations will be set in individual files, i.e. as virtual sites, then they will be enabled/disabled when needed.
1-Creating SVN Repository:
cd /var sudo mkdir MyProjects cd MyProjects sudo mkdir svn cd svn sudo svnadmin create SVN_Project_01
sudo chown -R www-data.www-data /var/MyProjects
2-Setting Up Trac
cd /var/MyProjects/ sudo mkdir trac cd trac sudo trac-admin Trac_Project_01 initenv -->Project Name [My Project]> -->Database connection string [sqlite:db/trac.db] (choose defaults) -->Repository type [svn]> -->Path to repository [/path/to/repos]> /var/MyProjects/svn/SVN_Project_01
sudo chown -R www-data.www-data /var/MyProjects
sudo trac-admin /var/MyProjects/trac/Trac_Project_01 permission add username MILESTONE_ADMIN REPORT_ADMIN ROADMAP_ADMIN TICKET_ADMIN TRAC_ADMIN
Setting Up Apache:
At this stage, we need to secure the web access to the repository so only authorized persons who can modify it, enable the SSL mode so connection is encrypted, and modify apache configurations accordingly:
Securing Web Access
- Enable the Authenticated Access by adding a username and password:
sudo htpasswd -cb /etc/apache2/passwords username password
- Enabling the SSL Module:
Refer to this blog: How-To Enable SSL on Apache2 Server
Note: You will need to download the Trac package and link to the cgi-bin/trac.cgi and cgi-bin/trac.fcgi in the configuration.
In my case, this the path to these files after extracting:
Create the following files or you may create one of them:
Track through CGI
cd /etc/apache2/sites-available/ cat > svn.cgi
##SVN <Location /MyProjects/svn/SVN_Project_01> DAV svn SVNPath /var/MyProjects/svn/SVN_Project_01 AuthType Basic AuthName "Subversion Repository - SVN_Project_01" AuthUserFile /etc/apache2/.htpasswd # <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user SSLRequireSSL # </LimitExcept> </Location> ##Trac ScriptAlias /MyProjects/trac /usr/local/Trac-0.11.6/cgi-bin/trac.cgi <Location /MyProjects/trac> SetEnv TRAC_ENV_PARENT_DIR /var/MyProjects/trac </Location> <Location "/MyProjects/trac"> SSLRequireSSL AuthType Basic AuthName "Trac Login for Projectname Website" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Location> <Location "/MyProjects/trac/Trac_Project_01/login"> SSLRequireSSL AuthType Basic AuthName "Trac Login for Projectname Website" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Location>
Track through FCGI
cd /etc/apache2/sites-available/ cat > svn.fcgi
##SVN <Location /MyProjects/svn/SVN_Project_01> DAV svn SVNPath /var/MyProjects/svn/SVN_Project_01 AuthType Basic AuthName "Subversion Repository - SVN_Project_01" AuthUserFile /etc/apache2/.htpasswd # <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user SSLRequireSSL # </LimitExcept> </Location> ##Trac ScriptAlias /MyProjects/trac /usr/local/Trac-0.11.6/cgi-bin/trac.fcgi ##fastcgi #FastCgiConfig -initial-env TRAC_ENV=/var/MyProjects/trac/Trac_Project_01 #FastCgiConfig -initial-env TRAC_ENV_PARENT_DIR=/var/MyProjects/trac ##fcgi DefaultInitEnv TRAC_ENV_PARENT_DIR /var/MyProjects/trac #<Location "/MyProjects/trac"> ##fastcgi #SetEnv TRAC_ENV_PARENT_DIR "/var/MyProjects/trac" #SetEnv TRAC_ENV "/var/MyProjects/trac/Trac_Project_01" #AddHandler fastcgi-script .fcgi #AuthType Basic #AuthName "Trac Repository - Trac_Project_01" #AuthUserFile /etc/apache2/.htpasswd #Require valid-user #SSLRequireSSL #</Location> <Location "/MyProjects/trac/Trac_Project_01/login"> SSLRequireSSL AuthType Basic AuthName "Trac Login for Projectname Website" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Location>
Track through FastCGI
cd /etc/apache2/sites-available/ cat > svn.fastcgi
##SVN <Location /MyProjects/svn/SVN_Project_01> DAV svn SVNPath /var/MyProjects/svn/SVN_Project_01 AuthType Basic AuthName "Subversion Repository - SVN_Project_01" AuthUserFile /etc/apache2/.htpasswd # <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user SSLRequireSSL # </LimitExcept> </Location> ##Trac ScriptAlias /MyProjects/trac /usr/local/Trac-0.11.6/cgi-bin/trac.fcgi #FastCgiConfig -initial-env TRAC_ENV=/var/MyProjects/trac/Trac_Project_01 FastCgiConfig -initial-env TRAC_ENV_PARENT_DIR=/var/MyProjects/trac #DefaultInitEnv TRAC_ENV /var/MyProjects/trac/Trac_Project_01 <Location "/MyProjects/trac"> #SetEnv TRAC_ENV_PARENT_DIR "/var/MyProjects/trac" #SetEnv TRAC_ENV "/var/MyProjects/trac/Trac_Project_01" AddHandler fastcgi-script .fcgi #AuthType Basic #AuthName "Subversion Repository - SVN_Project_01" #AuthUserFile /etc/apache2/.htpasswd #Require valid-user #SSLRequireSSL </Location> <Location "/MyProjects/trac/Trac_Project_01/login"> SSLRequireSSL AuthType Basic AuthName "Trac Login for Projectname Website" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Location>
Track through Python
cd /etc/apache2/sites-available/ cat > svn.python
##SVN <Location /MyProjects/svn/SVN_Project_01> DAV svn SVNPath /var/MyProjects/svn/SVN_Project_01 AuthType Basic AuthName "Subversion Repository - SVN_Project_01" AuthUserFile /etc/apache2/.htpasswd # <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user SSLRequireSSL # </LimitExcept> </Location> ##Trac <Location /MyProjects/trac> SetHandler mod_python PythonInterpreter main_interpreter PythonHandler trac.web.modpython_frontend PythonOption TracEnvParentDir /var/MyProjects/trac PythonOption TracUriRoot /MyProjects/trac #SSLRequireSSL #AuthType Basic #AuthName "Trac Login for Projectname Website" #AuthUserFile /etc/apache2/.htpasswd #Require valid-user </Location> <Location "/MyProjects/trac/Trac_Project_01/login"> SSLRequireSSL AuthType Basic AuthName "Trac Login for Projectname Website" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Location>
After that, you will need to enable ONLY one of them:
sudo a2ensite svn.cgi
sudo a2ensite svn.fcgi
sudo a2ensite svn.fastcgi
sudo a2ensite svn.python
Finally, reload apache server:
sudo /etc/init.d/apache2 reload
You will need to configure your prefered IDE with the created SVN repository if it supports the synchronization with SVN repositories. In my case as a Java/Java EE Developer, I’ ll refer to the most known open IDEs which are: NetBeans and Eclipse. SVN is just a little part of them to control versioning!
Hence, this would help you get a well managed and organized development environment!
Why to Enable SSL Mode?
Without digressing too much in the Security topic, SSL (Secure Socket Layer) is a cryptographic protocol that provides security for communications over networks such as the Internet. Therefore, you will need such a protocol to protect your connection with the web server by encrypting the information being exchanged. So, if a sniffer drops into the connection then that information is not compromised. e.g. remotely logging in to your machine requires from you to submit your username and password. If the connection is not encrypted using one of the cryptographic protocols then such must-be-kept-secret information is exposed and imagine the impact on your system if such info was in somebody’s hand!
So, let’s discuss the steps of how to enable the SSL mode:
- Generate a Self-Signed Certificate
cd /etc/apache2/ sudo mkdir certs cd ./certs sudo openssl req -new -x509 -nodes -days 365 -out server.crt -keyout server.key
- Encrypt the Private Key (Optional)
The way of doing it is by passing a “passphrase“:
sudo openssl rsa -des3 -in server.key -out server.key
Note: I tend not to do this step due to the fact that when Apache2 is restarted you will be asked to type the passphrase again. Therefore, I just change the key and certificate files permission so they are only read by Apache2!
- Enable the SSL Modules
You can either enable the SSL Module by running these commands:
sudo a2enmod ssl sudo /etc/init.d/apache2 restart
OR if you are curious about what it does you can do the following steps instead:
cd /etc/apache2/mods-enabled sudo ln -s /etc/apache2/mods-available/ssl.load ./ sudo /etc/init.d/apache2 restart
- Create the SSL Site
sudo pico /etc/apache2/sites-available/MySSL
################################## ##-->@Author Husain Al-Khamis<--## ################################## <VirtualHost "*:443"> ServerAdmin webmaster@localhost DocumentRoot /var/www ##-->Me<--## SSLEngine on SSLCertificateFile /etc/apache2/certs/server.crt SSLCertificateKeyFile /etc/apache2/certs/server.key ##-->Me<--## <Directory "/"> Options FollowSymLinks AllowOverride None </Directory> <Directory "/var/www/"> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 ::1/128 </Directory>
- Listen on Port 443
sudo pico /etc/apache2/ports.conf
And add the following:
<IfModule mod_ssl.c> # SSL name based virtual hosts are not yet supported, therefore no # NameVirtualHost statement here NameVirtualHost *:443 Listen 443 </IfModule>
- Enable the SSL Site
Same as in step# 3, you can either perform these commands
sudo a2ensite MySSL sudo /etc/init.d/apache2 restart
OR alternatively, you can do it manually in this way:
sudo pico /etc/apache2/apache2.conf
And add the following to the end of it:
# Include the secured host configurations: Include /etc/apache2/sites-available/MySSL
Then, Restart Apache2:
sudo /etc/init.d/apache2 restart
So, have a secure web surfing!
Upon upgrading to Ubuntu 9.10 Karmic Koala, some including me noticed that any Flash-containing website like YouTube.com then you can’t click on that Flash object. In other words, Flash is not recognizing mouse clicks.
It appeared that this was a bug that was reported here.
A workaround for such a bug has been put in place. So here are the workarounds as the official bug tracker suggested:
- WORKAROUND 1: Disable compiz
- WORKAROUND 2: Remove flashplugin-nonfree / flashplugin-installer and install from adobe
- WORKAROUND 3, and this is the one worked with me:
- Open a terminal, or you can press Alt + F2, and type
- gksudo gedit /usr/lib/nspluginwrapper/i386/linux/npviewer
- Then add “export GDK_NATIVE_WINDOWS=1” before the last line of text.
- Restart your FireFox, or any application uses Flash.
So, This is it as far as I know!
I’ve just started blogging seriously. Many free blogging tool providers are available. Widely known are the WordPress and Blogger. For some reasons, I decided to use WordPress as my main blogging tool.
I’ve given some thoughts to start developing a Web-based Blogging Tool just to have some hands-on experience on the Web Development and Design process. Besides, as I advocate the Open Source Community I would like to make a humble contribution to this great community.
I was wondering which platform on which I should rely to start developing this project. From my notice, most web applications nowadays are developed in PHP. PHP has been used very widely by the public. Well, PHP is a very cool scripting language and I’ve always looked forward to learning it. But, as it is a top priority to my career path is to have a solid development background using Java/J2EE technology, I’ve decided to start a project for developing a Java Web-based Blogging Application. Fortunately, somebody already started developing such a project long time ago, since 2002. This application is sponsored by Apache Software Foundation and is called Roller Weblogger.
So, trying to make Roller up and running, we need to download a Java Web Server to be the deploying environment for Roller. You can download Tomcat and use it as a Web Server to deploy Roller. Refer to this Installation Guide to deploy Roller on Tomcat.
In my case, I decided to use GlassfishV3 due to the features and frameworks bundled with. Here, I’ll explain step by step how to install GlassfishV3, deploy Roller on it, and avoid a possible problem that might hinder the deployment process!
So, let’s start!
STEP 1: GlassfishV3
STEP 1.1: Download GlassfishV3
Download GlassFishV3 Prelude or Preview. Compare between them here.
Refer to Comment#:1
STEP 1.2: Install/Configure GlassfishV3
Install GlassfishV3 by simply running the script. It is not a big deal just follow the instructions!
STEP 1.3: Start GlassfishV3
Go to where Glassfish binary files are:
- To start the domain:
- To stop the domain:
Or alternatively, I prefer to start/stop the server by running these: “log will be output to the screen, so you can monitor it”
- To start the server:
- To stop server:
Now, you can reach the Admin Console at:
For further details, have a look at the GlassfishV3 Quick Start Guide!
STEP 2: MySQL JDBC Connection
STEP 2.1: Download Connector/J
You will need to download the MySQL JDBC Connector to allow Roller interact with the MySQL Database Server. Here, I used the MySQL Connector/J 5.1.10.
Place the Connector/J in:
STEP 2.2: Provide JNDI
- Create JDBC Connection Pool: (Admin GUI -> Application Server -> Resources -> JDBC -> Connection Pools)
- Create JDBC Resources (Admin GUI -> Application Server -> Resources -> JDBC -> JDBC Resources)
Restart Glassfish and Ping the JDBC Connection!
STEP 3: Roller 4
STEP 3.1: Download Roller
STEP 3.2: Create Database
STEP 3.3: Create roller-custom.properties
To connect to the MySQL database you should create the roller-custom.properties file in either 2 ways “as far as I noticed“:
- Roller will interact with database Directly:
Place it in:
- Roller will interact with database through the JNDI:
Place it in:
Make sure that the following Descriptor is placed in this directory:
STEP 3.4: Deploy Roller
Deployment must be easy and straightforward too from the Admin GUI, or You can place the $ROLLER_DIR in the autodeploy directory if you want just to try it out.
STEP 4: A Problem To Be Avoid
I only got Roller and GlassFishV3 working together by disabling the following JPA configuration and setting its flag to false and then adding it to “roller-custom.properties“:
Upon upgrading to Ubuntu 9.10 I noticed that some applications won’t run any more and the problem was apparently related to the dependency library libstdc++5. This library was deprecated and libstdc++6 was used instead. It appeared that the removal of this library wasn’t wise, and it was reported as a bug which can be found here. You might see an Error Message like this when try to run an application depends on the missing library:
“Error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory“
To me, the packages that were affected are:
- Thunderbird (downloaded from Mozilla website)
- GWT (Google Web Toolkit) in the Hosted Browser mode
A workaround of this bug has been put in place. Mine is almost similar:
Since libstdc++5 was discontinued in Ubuntu 9.10 (Karmic Koala) then we need to download the Ubuntu 9.04 (Jaunty Jackalope) package.
Note: You will need the i386 package if you use Ubunut_i386, and you will need both i386 & amd64 packages if you use the Ubuntu_amd64 as there are i386 applications run under the amd64 architecture!
For Ubuntu 9.10 i386
Download the i386 Jaunty libstdc++5 package.
and simply install it by double click on it, or through the command line you can run this:
For Ubuntu 9.10 amd64
As I said before we need to download both the i386 & amd64 packages:
The amd64 package:
as simple as we did for Ubuntu_i386.
Download the amd64 Jaunty libstdc++5 package.
and simply install it by double click on it, or through the command line you can run this:
The i386 package:
Download the i386 Jaunty libstdc++5 package.
Here, we need to extract the libstdc++5 library ourself as this deb package is an i386 one. So, if you try to double click on it or install from the command line it will give you an error message “Wrong architecture ‘i386′”
The way to solve this is to extract the libraries yourself and place them in the correct directory. Starting from the director where the package has been downloaded to, run the following:
Up to here, this must solve your problem with the libstdc++5. Stick with it till Ubuntu community find a solution for this bug.
5 days ago, I wrote an article narrating my Linux story “The Greatest Impact in My Life: Linux“. This article was part of the contest held at BeginLinux.com website. The contest was about writing an article discussing “Why Are You Thankful For Linux?”
The good news is that: I was announced the WINNER of this contest! So, I’m so glad for this and I wish the best of luck for the rest of us in the upcoming contests.
Here, I would like to tell that I’m not glad because I’m the WINNER but rather I’m glad, happy, and delighted because actually I participated in a contest, I presented a work, I made an effort, I had to research, I got the acquaintance of a new community, I expanded my social network, I was connected to new fan of Linux, and countless benefits of participating in a contest!
So in case I was:
- The winner then that would increase my self-confidence.
- Not that winner then that would let me ask myself why I wasn’t that winner, what my weaknesses are, what the winner has that I don’t …etc. and that will encourage me to improve myself and strengthen my weaknesses! and of course, I would be so happy for the winner!
Therefore, I would like to thank the BeginLinux.com moderators who came up with the contest very much, and would like to thank every member participated and presented his/her idea in this contest.
Looking forward to more participations.