Shell Scripting

A collection of shell scripting resources.

Shell scripts are plain text files with an .sh extension.

If permission denied, the shell script may just need to be made executable. Example: for a script named backup.sh, using chmod u+x backup.sh will grant only the owner of the file execution permissions. For all users, replace the u with an a. You could also use chmod +x backup.sh which is the same as chmod a+x backup.sh.

chmod u+x backup.sh

Then try running the script again.

./backup.sh

rsync

This OS X example uses rsync to backup the Documents folder on the hard drive to an existing Documents folder on a Lexar USB stick. The --delete-before option deletes files from the target that do not exist in the source before rsync.

if [ -d "/Volumes/Lexar/Documents" ];
then
    rsync -avP "/Volumes/Macintosh HD/Users/Woz/Documents/" "/Volumes/Lexar/Documents/" --delete-before
fi

Use the --exclude option to prevent specific files and folders from being synced. This example excludes .DS_Store files, .git and node_modules folders from the backup.

if [ -d "/Volumes/Lexar/Code" ];
then
    rsync -avP --exclude .DS_Store --exclude .git --exclude node_modules "/Volumes/Macintosh HD/Users/Dinesh/Code/" "/Volumes/Lexar/Code/"
fi
  • These shell script examples are written for Cygwin and /cygdrive/c is how the C: drive of Windows is accessed. For Cygwin, the rsync package would need to be installed. On OS X and Linux, rsync is usually included in the system install.

This example uses rsync to backup an existing file to a server share.

if [ -f /cygdrive/c/Users/Gilfoyle/Dropbox/Private/Hooli.xlsx ];
then
    rsync -avP /cygdrive/c/Users/Gilfoyle/Dropbox/Private/Hooli.xlsx //FILESERVER/share/Hooli.xlsx
fi

This example uses rsync to backup an existing Private Dropbox folder to a server share.

if [ -d /cygdrive/c/Users/Gilfoyle/Dropbox/Private ];
then
    rsync -avP /cygdrive/c/Users/Gilfoyle/Dropbox/Private/ //FILESERVER/share/
fi

tar

This example checks for the existing source directory and creates a compressed archive in the current directory with a datetime string included in the filename. The source directory and its contents will be added to the archive while exluding node_modules .git and sass.cache

if [ -d /cygdrive/c/Users/Dinesh/Code ];
then
    now=$(date +"%Y-%m-%dT%H%M%S")
    tar -zcvf code_$now.tar.gz --exclude .git --exclude node_modules --exclude sass.cache /cygdrive/c/Users/Dinesh/Code
fi

dos2unix

Find all files inside the current directory and execute dos2unix on each converting DOS format line breaks (CRLF) to Unix format (LF).

find . -type f -exec dos2unix {} \;

unix2dos

Find all files inside the current directory and execute unix2dos on each converting Unix format line breaks (LF) to DOS format (CRLF).

find . -type f -exec unix2dos {} \;

Resources

Virtual Machine for LAMP Development

This blog post addresses a common scenario — a local LAMP development environment on your Windows or OS X computer. You could use XAMPP on either Windows or OS X, MAMP on OS X or native Apache in OS X. Here, we will create a Linux virtual machine with Samba configured to share the Virtual Machine file system with the host computer. We will also create virtual hosts, install and configure WordPress and Xdebug.

First thing you will need is Virtual Machine software. For my Virtual Machine, I chose to install the free VMware Player. If your host operating system is OS X, you can install VirtualBox if you do not want to purchase VMware Fusion or Parallels. You could also use VirtualBox on a Windows host instead of VMware if you prefer.

Next, you will need to create a Linux virtual machine. I decided to Download Ubuntu Server 32 bit since it is compatible with the intel processor of my host computer.

Install SSH

After creating a new VM and installing Linux, time to get our dev environment setup. Install SSH so we can run commands from the host. Later we will use ssh to to tunnel the Xdebug connection from the VM back to the host.

# update the distro
$ apt-get update
$ apt-get upgrade -y

# install SSH
$ sudo apt-get -y install openssh-client openssh-server

Get the IP address of the Linux virtual machine.

# lookup inet addr ip 
$ ifconfig eth0

From the host, open a terminal and SSH into the virtual machine. In a Windows host, you can use Cygwin, PuTTY or a Git Bash.

$ ssh jim@192.168.59.129
...
jim@192.168.59.129's password:
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic i686)

 * Documentation:  https://help.ubuntu.com/
Last login: Sat Nov  2 12:55:45 2013
jim@ubuntu:~$

Install LAMP server

Read my blog post for more on how to install the LAMP server and phpMyAdmin: Mint LAMP Development Environment

# install LAMP server 
$ sudo apt-get install lamp-server^

Restart Apache and test the web server root URL from a host browser using the virtual machines IP address, e.g., http://192.168.59.129/.

$ sudo /etc/init.d/apache2 restart

Virtual Hosts

# edit the Apache config
$ sudo nano /etc/apache2/apache2.conf

# in apache2.conf, add these virtual host settings below the files node
NameVirtualHost *:80

<VirtualHost *:80>
   DocumentRoot /var/www
   ServerName www.ubuntu.vm
   ServerAlias ubuntu.vm
</VirtualHost>

<VirtualHost *:80>
   DocumentRoot /var/www/wordpress
   ServerName www.wordpress.vm
   ServerAlias wordpress.vm
</VirtualHost>

Edit your hosts file in your host operating system to map to the virtual hosts you specified in the Apache config of the Linux virtual machine.

192.168.59.129  ubuntu.vm
192.168.59.129  wordpress.vm

Since we have set the hosts file to map 192.168.59.129 to ubuntu.vm, once you disconnect your current ssh session (Ctrl+D), you will need to use the host alias you specified in the hosts file to reconnect.

ssh jim@ubuntu.vm

Samba Network Share

You can use Samba to share the /var folder in you virtual machine.

# install Samba 
$ sudo apt-get install samba

Linux system permissions take precedence over Samba permissions. For example if a directory does not have Linux write permission, setting samba writeable = Yes will not allow to write to shared directory / share.

# edit Samba config 
$ sudo nano /etc/samba/smb.conf

####### Authentication #######
  security = user

# bottom of smb.conf
[etc]
    path = /etc
    browsable = yes 
    read only = no 
    create mask = 0755 
    admin users = jim

[home]
    path = /home
    browsable = yes 
    read only = no 
    create mask = 0755 
    admin users = jim

[var]
    path = /var
    browsable = yes 
    read only = no 
    create mask = 0755 
    admin users = jim 

Ctrl+C to exit nano, press Y to save and return to write your changes to the file.

Install the PAM authentication module for Samba which will sync the system users to the Samba user database.

# Install libpam-smbpass package
$ sudo apt-get install libpam-smbpass

# restart Samba for the new settings to take effect
$ sudo restart smbd
$ sudo restart nmbd

# set user jim as the owner of share directory
$ sudo chown -R jim /var/www

For OS X, open Finder, select Go > Connect to Server (command + K). Enter the server address: smb://ubuntu.vm and select Connect

OS X - Connect to Server Dialog
OS X – Connect to Server Dialog

For Windows, select Run from the Start menu or open Windows Explorer and enter:

\\ubuntu.vm\www
Windows Run Dialog
Windows Run Dialog

Server Time Synchronisation

Server time is likely to become inaccurate in our development scenario. We can use Ubuntu’s NTP server and NTP’s server pool to keep the server datetime accurate.

# install ntp
$ sudo apt-get install ntp

# edit /etc/ntp.conf
$ sudo nano /etc/ntp.conf
# add these two lines
server ntp.ubuntu.com
server pool.ntp.org

# change server time zone
$ sudo dpkg-reconfigure tzdata

Enabling Module Rewrite

The mod_rewrite Apache module is a rule-based rewriting engine to rewrite requested URLs. It is very useful for genarating SEO friendly URLs.

# enable the module 
$ sudo a2enmod rewrite

# restart apache to activate the configuration change
$ sudo service apache2 restart

Install WordPress

# download the latest WordPress to a downloads directory
$ mkdir ~/downloads
$ cd ~/downloads

$ wget http://wordpress.org/latest.tar.gz

# -z : Uncompress the resulting archive with gzip command.
# -x : Extract to disk from the archive.
# -v : Produce verbose output i.e. show progress and file names while extracting files.
# -f latest.tar.gz : Read the archive from the specified file.
# -C : To change the directory (current directory is default)

# this will create /var/www/wordpress since the archive root directory is wordpress 
$ sudo tar -zxvf latest.tar.gz -C /var/www

# make yourself the owner
sudo chown jim /var/www/wordpress

Create Database

Create MySql database, user and grant permissions in one line.
base_user is typically root
base_user_pass is password for base_user (no space between -p and base_user_pass is important)

mysql -u base_user -pbase_user_pass -e "create database new_db; GRANT ALL PRIVILEGES ON new_db.* TO new_db_user@localhost IDENTIFIED BY 'new_db_user_pass'"

phpMyAdmin

If phpMyAdmin is installed, you can use it to create the wordpress database as described in this next section. Read my earlier blog post if you need help doing this, Mint LAMP Development Environment

In the browser, goto http://ubuntu.vm/phpmyadmin/. Select the Databases tab and under “Create new database”, enter in a database name such as “wordpress”, and select Create. Collation will automatically be assigned by MySQL when the database tables are created, during the WordPress installation.

# copy WordPress sample config
$ cd /var/www/wordpress
$ cp wp-config-sample.php wp-config.php

In the browser, goto http://wordpress.vm and start the installation. If you need to manually edit the wp-config.php file, you can do so with nano.

$ nano wp-config.php 

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'database_name_here');

/** MySQL database username */
define('DB_USER', 'username_here');

/** MySQL database password */
define('DB_PASSWORD', 'password_here');

/** MySQL hostname */
define('DB_HOST', 'localhost');

After saving wp-config, complete the WordPress installation at http://wordpress.vm

You can backup the MySql database using mysqldump within your ssh session. For example, replace mysql_username with your username and mysql_password with your password:

$ mysqldump -h localhost -u mysql_username -pmysql_password wordpress > /home/wordpress-$(date +"%Y-%m-%d").sql

WordPress Revisions Cleanup

Run this sql query in phpMyAdmin to delete all revisions from each post, including all of its meta data.

DELETE a,b,c
FROM wp_posts a
LEFT JOIN wp_term_relationships b ON (a.ID = b.object_id)
LEFT JOIN wp_postmeta c ON (a.ID = c.post_id)
WHERE a.post_type = 'revision'

Xdebug

$ sudo apt-get install php5-xdebug

Check Xdebug

$ php --version

PHP 5.3.10-1ubuntu3.8 with Suhosin-Patch (cli) (built: Sep  4 2013 20:05:42)
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2012 Zend Technologies
    with Xdebug v2.1.0, Copyright (c) 2002-2010, by Derick Rethans

Configure Xdebug

$ sudo nano /etc/php5/conf.d/xdebug.ini

Add the following xdebug config values after the zend_extension path value so it looks like this.

zend_extension=/usr/lib/php5/20090626+lfs/xdebug.so
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_host=127.0.0.1
xdebug.remote_port=9000

When debugging from the host computer, use ssh to tunnel the debug-connection back to it.

# ssh remote port forwarding
ssh -R 9000:localhost:9000 jim@ubuntu.vm

Using the “-R” argument tells ssh to listen on TCP port 9000 on the ubuntu server and forward it back through the ssh connection to the TCP port specified (localhost:9000). So from Xdebug’s perspective on the Ubuntu server, it is talking to localhost on the local computer.

More info: Mint LAMP Development Environment – Part Two | Xdebug

Other PHP debugging options

You could use the FirePHP Extension for Firefox to log to your Firebug Console using a simple PHP method call. There is also a SublimeTextXdebug package for Sublime Text.

If all you have is ssh and nano

<?php var_dump(__FILE__, __LINE__, $_REQUEST); ?>
or
<?php
echo '<pre>';
print_r( $somevariable );
echo '</pre>';
?>

Resources

Mint LAMP Development Environment – Part Two

Linux Mint 15 ‘Olivia’

I decided it was time to ditch my Linux 13 KDE setup and try the improved MATE 1.6 desktop now that Linux Mint 15 released. I find it pretty slick and fast, a nice change from Windows when I decide to do some development in a true LAMP environment. Speaking of which, we need to get that installed. Before doing that, the first thing I like to do is open up a terminal and run:

sudo apt-get update

This will download the package lists from the repositories and update them with information on the latest package versions. If you have not installed LAMP (Linux, Apache, MySQL & PHP) please read my earlier blog post, Mint LAMP Development Environment. Then you can return to this post to add an IDE with debugging capabilities along with some other nice tools.

Xdebug

sudo apt-get install php5-xdebug

Check Xdebug

php --version
PHP 5.4.9-4ubuntu2 (cli) (built: Mar 11 2013 16:09:26) 
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies
    with Xdebug v2.2.1, Copyright (c) 2002-2012, by Derick Rethans

Configure Xdebug

sudo pluma /etc/php5/mods-available/xdebug.ini

Add the following xdebug config values after the zend_extension path value so it looks like this.

zend_extension=/usr/lib/php5/20100525/xdebug.so
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_host=127.0.0.1
xdebug.remote_port=9000

NetBeans

The IDE I prefer for Drupal development as of this writing is NetBeans. I found this post, Configuring NetBeans for Drupal development to be very helpful. In the Usage section of the post, the author suggests using the sites folder of your Drupal installation when creating the New Project > PHP Application with Existing Sources. This did not work for me, instead, I selected the root folder of my Drupal install as the Sources Folder for the NetBeans IDE project import. By including the entire Drupal installation in my NetBeans project, I could step through and debug any of the Drupal code including modules in the sites folder.

NetBeans IDE Resources

Eclipse

Download

As of this writing, the latest stable version of Eclipse was Eclipse Juno (4.2). I downloaded the Eclipse Classic 4.2.2 64 bit Linux version from http://www.eclipse.org/downloads/. Make sure you download the appropriate 32 or 64 bit version of Eclipse to match your Linux Mint operating system. To check which version of Linux Mint you are running, in a terminal, enter

uname -m

i686 = 32-bit; x86_64 = 64 bit

Install

Open up a terminal, change to the Downloads directory and and extract the archive.

cd ~/Downloads
tar xzf eclipse-SDK-4.2.2-linux-gtk-x86_64.tar.gz

With superuser permissions, move the extracted eclipse directory and its contents into the /opt/ folder.

sudo mv eclipse /opt/

Create a symbolic link so you can start Eclipse from a terminal

sudo ln -s /opt/eclipse/eclipse /usr/bin/eclipse

Launch eclipse

eclipse

or if working on files directly in /var/www/

sudo eclipse

PHP Development Tools (PDT)

To install latest stable PDT build in Eclipse, select Help > Install New Software and enter following URL in the work with input:
http://download.eclipse.org/tools/pdt/updates/release

Recoll

You could use the terminal to search for files with commands such as find, grep, and awk. However, I like the the Recoll desktop search tool.

Filezilla

When you are ready to deploy your development work to a remote server via FTP or SFTP, this cross-platform FTP client is my goto tool for the job.

sudo apt-get install filezilla

Resources

Magento Development Environment

Requirements

Requirements Test

Does your server meet the requirements for Magento?

Install Sample Data

Before installing Magento, download and install sample data and media.
  1. Using phpMyAdmin, import the sample data sql file “magento_sample_data_for_1.6.0.0.sql” into an empty magento database.
  2. Copy the contents of the sample “media” folder into the media folder of your Magento installation.

Admin login fails without error message on localhost

On or near line 104 of /var/www/magento/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php
/* this is commented out for locahost development versions so admin can login

if (isset($cookieParams['domain'])) {
	$cookieParams['domain'] = $cookie->getDomain();
}
*/
Add write permissions to:
~/magento/app/etc/
~/magento/media/
~/magento/var/

Resources

Mint LAMP Development Environment

OPERATING SYSTEM: Linux Mint 13 Maya

For Drupal development, since I already have a Linux Mint system setup, I decide to focus on it rather than the Windows 8 Acquia Drupal setup I blogged about a few days ago.

Apache, MySQL & PHP

Install all of these applications with a single command. Thanks to Unix System Engineer, Nitin Sookun for posting this on the Linux Mint Community website.

Since you will need root permissions when performing write operations outisde of your home directory, the commands shown below presume that you are logged in as root or are in a root instance of the terminal. If you are not in a root terminal, you can prepend the commands with ‘sudo’, for example listing the contents of the current directory:

sudo ls
Sudo Sandwich
Sandwich

In a root terminal (such as ‘Konsole as root’).

apt-get install lamp-server

If you get this error message (as I did during a Mint Xfce LAMP install)

E: Unable to locate package lamp-server

Then you will need a carat at the end:

apt-get install lamp-server^

Then install phpMyAdmin

apt-get install phpmyadmin

Restart Apache and try loading http://localhost/phpmyadmin.

/etc/init.d/apache2 restart

If phpMyAdmin doesn’t load, try adding an Include to your Apache config with a text editor (such as ‘Kate’). In your terminal, open a new bash shell tab, and launch the editor (gedit, kate, etc.)

kate /etc/apache2/apache2.conf

In the apache configuration file, near the bottom, look for the location of other Include statements and insert this phpmyadmin include as needed.

Include /etc/phpmyadmin/apache.conf
/etc/init.d/apache2 restart

mySQL

If you need to set or change the mySQL root password, use this command syntax when the password has not been set.

$ mysqladmin -u root password NEWPASSWORD

Use this command syntax to change the root password.

$ mysqladmin -u root -p'OLDPASSWORD' password 'NEWPASSWORD'

Drupal

Download the latest stable Drupal core distro from http://drupal.org/download. Extract the downloaded file, for example, drupal-7.19.tar.gz to the host root, /var/www. Then with the mv command, rename the drupal-7.19 directory extracted into www to drupal7

tar -xf drupal-7.19.tar.gz -C /var/www
mv /var/www/drupal-7.19 /var/www/drupal7

Copy the default.settings.php to settings.php. Then apply write permissions to both the dafault directory and the settings.php file so Drupal can modify settings.php as needed during the install.

cd /var/www/drupal7/sites
cp default/default.settings.php default/settings.php
chmod a+w default
chmod a+w default/settings.php

Note, after Drupal is installed, reset settings.php file permissions to read only using a-w

Virtual Hosts

Setting up some Virtual Hosts in our Apache config. In terminal, open the apache cofig file for editing (gedit, kate, etc.)

kate /etc/apache2/apache2.conf

Below the Files node, add this code block.

NameVirtualHost *:80

  ServerName default
  DocumentRoot /var/www/

  ServerName drupal7
  DocumentRoot /var/www/drupal7

  ServerName drupal8
  DocumentRoot /var/www/drupal8

Restart Apache to load our config changes

/etc/init.d/apache2 restart

Hosts file

Add the two named virtual hosts to the hosts file

kate /etc/hosts
127.0.1.7	drupal7
127.0.1.8	drupal8

Restart apache …

/etc/init.d/apache2 restart

Now you can access the new Drupal website at http://drupal7

Next – Mint LAMP Development Environment – Part Two

Resources

Improved Font Rendering in Fedora 15 / Gnome 3

  1. Checkout this blog article at infinality.net: Subpixel Hinting and Other Enhancements for Freetype & Fedora 15 RPM Packages
  2. Per the infinality article, install the infinality-settings package
  3. Edit /etc/X11/.Xresources to read:
    Xft.autohint: 0
    Xft.lcdfilter:  lcddefault
    Xft.hintstyle:  hintfull
    Xft.hinting: 1
    Xft.antialias: 1
    Xft.dpi: 96
    Xft.rgba: rgb
  4. Install Windows fonts. To do this, I copied all the .ttf fonts from my mounted Windows 7 device (/mnt/ntfs/Windows/Fonts) to /usr/share/fonts.