Category Archives: Ubuntu/linux

related to unix/linux or ubuntu related findings

In Ansible is quite easy add users, pubkeys and other stuff to any server. If the project or organization have multiple servers, ssh-keys are a swiss-knife, but they are kinda difficult to handle if you create/destroy multiple servers every week.

Without using Ansible (or any other config management), to add or revoke access you will need to login in all servers by hand (or by a script) and this is the time taking process to do in all servers.

With Ansible we can setup a playbook to do all these things.

Let’s see how to use this with small example playbook.yml:


– hosts: webservers
gather_facts: false
remote_user: root
tasks:
– name: generating SSH keys
command: ssh-keygen -t rsa -f /root/.ssh/id_rsa -N ”
args:
creates: /root/.ssh/id_rsa

– name: fetch all public ssh keys
shell: cat ~/.ssh/id_rsa.pub
register: ssh_keys
tags:
– ssh

– name: check keys
debug: msg=”{{ ssh_keys.stdout }}”
tags:
– ssh

– name: deploy keys on all servers
authorized_key: user=root key=”{{ item[0] }}”
delegate_to: “{{ item[1] }}”
with_nested:
– “{{ ssh_keys.stdout }}”
– “{{groups[‘webservers’]}}”
tags:
– ssh

 

 

We can create an EC2 instances within  AWS using Ansible. Please find the below playbook to create two instances using Ansible.


– name: Provision an EC2 Instance
hosts: localhost
connection: local
gather_facts: False
tags: provisioning
# Necessary Variables for creating/provisioning the EC2 Instance
vars:
group_id:
– sg-3e82994d
volumes:
– device_name: /dev/sda1
volume_type: gp2
volume_size: 8
delete_on_termination: True
–  device_name: /dev/sda1
volume_type: gp2
volume_size: 8
delete_on_termination: True
tasks:
– name: Launch the first EC2 Instance
local_action: ec2
count=1
aws_access_key=XXXXXXXXXXXXXXXXXX
aws_secret_key=XXXXXXXXXXXXXXXXXXXXXXXXXX
group_id={{ group_id }}
instance_type=t2.micro
image=ami-cd0f5cb6
wait=false
region=us-east-1
keypair=MyKeyPair
vpc_subnet_id=subnet-2f3e894b
assign_public_ip=no
volumes={{ volumes }}
zone=us-east-1b
termination_protection=no
– name: Launch the second EC2 Instance
local_action: ec2
count=1
aws_access_key=XXXXXXXXXXXXXXXXXX
aws_secret_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
group_id={{ group_id }}
instance_type=t2.micro
image=ami-cd0f5cb6
wait=false
region=us-east-1
keypair=MyKeyPair
vpc_subnet_id=subnet-afaf8ca3
assign_public_ip=no
volumes={{ volumes }}
zone=us-east-1f
termination_protection=no
register: ec2
retries: “2”
delay: “3”
with_items: ec2.instances
no_log: False

 

 

The best way to get Ansible for Ubuntu is to add the project’s PPA to your system.

We need to install the software-properties-common package, which will give us ability to work with PPAs easily.

$ sudo apt-get update

$ sudo apt-get install software-properties-common

Once the package is installed we can add the Ansible PPA by typing th e following command.

$ sudo apt-add-repository ppa:ansible/ansible

press enter to accept PPA addition.

We need to refresh our system’s package index so that it is aware of the packages available in the PPA then we can install the software.

$ sudo apt-get update 

$ sudo apt-get install ansible

Set Up SSH Keys

Ansible primarily communicates with client computers through SSH.

Create a New SSH Key Pair

If you do not have already an SSH key pair that you would like to use for Ansible administration.

We can create the RSA keys by typing below command

$ ssh-keygen -t rsa

You will be asked to specify the file location of the created key pair, a passphrase, and the passphrase confirmation. Press ENTER through all of these to accept the default values.

Your new keys are available in your user’s ~/.ssh directory. The public key is called id_rsa.pub. The private key is called id_rsa.

We can add the ssh keys into client machine by typing

$ ssh-copy-id username@IP_Address

Configuring Ansible Hosts

Ansible keeps track of all of the servers that it knows about through a “hosts” file. We need to set up this file first before we can begin to communicate with our other computers.

Open the file with root privileges like this:

$ sudo vi /etc/ansible/hosts

[group_name]
alias ansible_ssh_host=your_server_ip

[webservers]
host1 ansible_ssh_host=192.0.2.1
host2 ansible_ssh_host=192.0.2.2
host3 ansible_ssh_host=192.0.2.3

We can put our configuration with YAML files start with “—“, so make sure you don’t forget that part.

Using Simple Ansible Commands

Now that we have our hosts set up and enough configuration details to allow us to successfully connect to our hosts, we can try out our very first command.

Ping all of the servers you configured by typing:

$ ansible -m ping all

Output:

host1 | success >> {
“changed”: false,
“ping”: “pong”
}

host3 | success >> {
“changed”: false,
“ping”: “pong”
}

host2 | success >> {
“changed”: false,
“ping”: “pong”
}

Overview

We will be setting up a Ruby on Rails development environment on Ubuntu 14.04.

The reason we’re going to be using Ubuntu is because the majority of code you write will run on a Linux server. Ubuntu is one of the easiest Linux distributions to use with lots of documentation so it’s a great one to start with.

Note: Before going to start installation please make sure that you have root privileges or not.

Installing Ruby Dependencies:The first step is to install some dependencies for Ruby.

   sudo apt-get update
   sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev     libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev
  

Next we’re going to be installing Ruby using rvm. You can install from source as well.

Install Ruby Using RVM:

    sudo apt-get install libgdbm-dev libncurses5-dev automake libtool bison libffi-dev
    gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
    curl -sSL https://get.rvm.io | bash -s stable
    source ~/.rvm/scripts/rvm
    rvm install 2.2.3
    rvm use 2.2.3 --default
    ruby -v
   

Installing Rails Dependencies:Since Rails ships with many dependencies, we’re going to need to install a Javascript runtime like NodeJS. This lets you use Coffeescript and the Asset Pipeline in Rails which combines and minifies your javascript to provide a faster production environment.

    curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash -
    sudo apt-get install -y nodejs
   

We can use various Rails versions with each Ruby by creating gemsets and then installing Rails within those using the normal gem commands:

    rvm gemset create starterapp
    rvm 2.2.3@starterapp
    gem install rails -v 4.2.4
   
  • “rvm gemset create starterapp” command is to create a gemset name called starterapp.
  • “rvm 2.2.3@starterapp” command is to specify Ruby version and our new gemset.
  • “gem install rails -v 4.2.4” command is to install specific Rails version.

Now that you’ve installed Rails, you can run the rails -v command to make sure you have everything installed correctly:

     rails -v
     # Rails 4.2.4
   

The last step is to install Bundler.

     gem install bundler
    

Configuring Git:(If you already installed git please ignore this step)
We’ll be using Git for our version control system so we’re going to set it up to match our Github account. If you don’t already have a Github account, make sure to register. It will come in handy for the future.

Replace my name and email address in the following steps with the ones you used for your Github account.

    git config --global color.ui true
    git config --global user.name "YOUR NAME"
    git config --global user.email "YOUR@EMAIL.com"
    ssh-keygen -t rsa -b 4096 -C "YOUR@EMAIL.com"
  

The next step is to take the newly generated SSH key and add it to your Github account. You want to copy and paste the output of the following command and paste it here

    cat ~/.ssh/id_rsa.pub
  

Once you’ve done this, you can check and see if it worked:

    ssh -T git@github.com
  

You should get a message like this:

    Hi excid3! You've successfully authenticated, but GitHub does not provide shell access.
  

Setting Up MySQL:

Rails ships with sqlite3 as the default database. Chances are you won’t want to use it because it’s stored as a simple file on disk. You’ll probably want something more robust like MySQL or PostgreSQL.If you’re coming from PHP, you may already be familiar with MySQL.

You can install MySQL server and client from the packages in the Ubuntu repository. As part of the installation process, you’ll set the password for the root user. This information will go into your Rails app’s database.yml file in the future.

    sudo apt-get install mysql-server mysql-client libmysqlclient-dev
  

Installing the libmysqlclient-dev gives you the necessary files to compile the mysql2 gem which is what Rails will use to connect to MySQL when you setup your Rails app.

Setting Up PostgreSQL:For PostgreSQL, we’re going to add a new repository to easily install a recent version of Postgres.

    sudo sh -c "echo 'deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main' > /etc/apt/sources.list.d/pgdg.list"
    wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | sudo apt-key add -
    sudo apt-get update
    sudo apt-get install postgresql-common
    sudo apt-get install postgresql-9.5 libpq-dev
  

The postgres installation doesn’t setup a user for you, so you’ll need to follow these steps to create a user with permission to create databases. Feel free to replace vijay with your username.

    sudo -u postgres createuser vijay -s

    # If you would like to set a password for the user, you can do the following
    sudo -u postgres psql
    postgres=# \password vijay
  

Final Steps: Let’s create your first Rails application:

    #### If you want to use SQLite (not recommended)
    rails new myapp

    #### If you want to use MySQL
    rails new myapp -d mysql

    #### If you want to use Postgres
    # Note that this will expect a postgres user with the same username
    # as your app, you may need to edit config/database.yml to match the
    # user you created earlier
    rails new myapp -d postgresql

    # Move into the application directory
    cd myapp

    # If you setup MySQL or Postgres with a username/password, modify the
    # config/database.yml file to contain the username/password that you specified

    # Create the database
    rake db:create

    rails server
  

Note: Sentence start’s with ‘#’ are comments not commands.

You can now visit http://localhost:3000 to view your new website. Now that you’ve got your machine setup, it’s time to start building some Rails applications.

If you received an error that said Access denied for user ‘root’@’localhost’ (using password: NO) then you need to update your config/database.yml file to match the database username and password.

Reference: gorails

Thanks for reading this Article. If you have any questions, feel free to post your comments and we’ll get back to you.

Configuring Network in OpenStack

* Open openstack dashboard and login

sys

* After successful login as the user admin you will end up in the admin view of the OpenStack Dashboard Horizon

sys

* Click on the Network Topology tab within the Project area

sys

Initially the Network Topology is empty, so we have to create a network for the VM’s
Here’s what we need at a base level:
1.An external (sometimes also called public) network that corresponds to the physical network segment providing us external connectivity.
2.An internal network per tenant to which we can attach VMs. These per tenant VM can use this internal network to communicate with each other. But also we don’t necessarily want to connect every VM to the outside world.
3.In order for the VMs, connected to the internal network, to reach the outside world (e.g. Internet) we also need a router providing Source NAT (SNAT) capability between internal and external network

* Return to the Admin view, choose the Networks tab and click on Create Network as below

sys

Give the new network a useful name such as Enetwork and tick the External Network box to designate it as an external network. You need to specify a project when you create a new network. Yet, an external network will be visible from all projects. It therefore doesn’t really matter which project you assign this network to. A good project to pick is the service project, as it is a core part of OpenStack

sys

* Next click on the network name – here Enetwork – to configure additional settings such as the subnet

sys

Within the Network Detail view, click on Create Subnet to associate a subnet with this network

sys

Give the Subnet as External and network address as 10.9.8.0/24 and gateway 10.9.8.249. Click on subnet Detail for next view

sys

Unselect the Enable DHCP checkbox as this is an external network which either already has an existing DHCP service available, or on which you don’t want OpenStack to supply DHCP capabilities. Specify the Allocation pools specific to your environment with an IP range that is not already in use within the selected subnet. Specify the DNS Name Servers and click Create to finalize the creation of the subnet

sys

Verify that the external network has been successfully created and is in the UP state

sys

Return to the tenant’s view and see how the previously created external network will look like. Choose the Project view, then click on the Network Topology tab. You can see the external network available to the tenant

sys

Next we will create the internal network. This task will be completed by the tenant within a project. Therefore still within project view, click on the Create Networks button in the above image

Enter internal as the Network Name and click on the Subnet to specify additional information as shown below

sys

Specify a Subnet Name as Internal along with the Network Address as 192.168.1.0/24 and Click on Subnet Detail for next view

sys

Enter the value of the DNS Name Server and finish the dialog with a click on Create

sys

The result is now an external network, which was created and is owned by the cloud administrator and an internal network, which was created and is owned by a project tenant. But we are not done yet: If we connect a workload to the internal network, it will obviously not have outbound connectivity as internal and external network are not connected. We can fix this by creating a router between the two. Click on Create Router to get started

sys

Give the Router a useful Router Name and finish the creation of the router with a click on Create Router

sys

Next we need to create the router’s interfaces on the two networks. Click on the router and choose view router details to get started

sys

Now click on Add Interface to add the internal interface first, As the Subnet choose the internal network that you created in an earlier step and click on Add interface to finish your selection

sys

Next we need to configure the external network as the upstream network. This is done by setting the gateway for the router.
Click on the Routers tab to leave the Router Detail view
Under Actions for the router click on Set Gateway

sys

As the External Network chose the network that is provided by the cloud operator and confirm the selection by clicking on Set Gateway

sys

Return to the Network Topology view to see the result

sys

Creating and Deploying a Virtual Machine

* From the Network Topology tab click on the Launch Instance button

sys

Give the instance name as FirstInstance, Flavour as m1.tiny, Instance count as 1, Instance Boot source as “Boot from Image” and Image name as “myFirstImage” and click on Networking for next view

sys

Move the network Internal from the Available networks pool to the Selected networks, by either clicking on the plus icon or using drag-and-drop and Click on Launch to finalize the creation of your VM instance.

sys

Wait for the new VM instance to be created and powered up. Once the Status indicates Active and the Power State shows running, The VM is up. Click on the name of the VM to see the Instance Detail.

sys

We can see information about the running VM instance – such as the ID, the status or the IP address. Click on the Console tab to connect to the VM

sys

Click on the Console tab to connect to the VM

sys

Via the Console you can access your VM instance, login to the server

sys

Ping the external network IP to verify the Network

sys

Configuring Nginx as reverse proxy SSL to tomcat

Install nginx
sudo apt-get install nginx

Do the below changes in tomcat $TOMCAT_HOME/conf/server.xml

Add adress=”127.0.0.1″, proxyname, proxy port as shown below

	<Connector address="127.0.0.1" port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443"
		proxyname="phonebook.nbostech.com"
		proxyport="443" />

Add under Host name=”localhost” appBase=”webapps”

<Valve className="org.apache.catalina.valves.RemoteIpValve" protocolHeader="X-Forwarded-Proto"  protocolHeaderHttpsValue="https" />

Now create site in Nginx:
Add the site as shown below under /etc/nginx/sites-enabled

upstream phonebook.nbostech.com {
           server 127.0.0.1:8080;
}

server {
 listen 80;
 server_name phonebook.nbostech.com;
 access_log /var/log/nginx/access.log;
 error_log /var/log/nginx/error.log;
 rewrite_log on;

  location /phonebook {
            proxy_pass http://phonebook.nbostech.com/phonebook;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_set_header Host $http_host;
             }

  location / {
            proxy_pass http://phonebook.nbostech.com/phonebook;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_set_header Host $http_host;
             }

      }

server {
   listen   443;
   server_name  phonebook.nbostech.com;
   access_log  /var/log/nginx/access.log;
   error_log /var/log/nginx/error.log;
   ssi on;
   ssl  on;
   ssl_certificate  /etc/ssl/certs/nbserver20.crt;
   ssl_certificate_key  /etc/ssl/certs/nbserver20.key;
   ssl_session_timeout  5m;
   ssl_protocols  SSLv2 SSLv3 TLSv1;
   ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
   ssl_prefer_server_ciphers   on;

  location  = / {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_set_header X-Forwarded-Proto https;
            proxy_connect_timeout      240;
            proxy_send_timeout         240;
            proxy_read_timeout         240;
            proxy_pass http://phonebook.nbostech.com/phonebook;
            }

  location /phonebook {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
	    proxy_set_header X-Forwarded-Proto https;
            proxy_connect_timeout      240;
            proxy_send_timeout         240;
            proxy_read_timeout         240;
            proxy_pass http://phonebook.nbostech.com/phonebook;
            }
       }

Restart the nginx and tomcat

LMS configure and install in ubuntu with Passenger Rails

1. Install Require Packages

2. Install RVM System Wide

3.Logout of the server and log back in

4. Install Ruby 1.8.7

5. Set the default Ruby version to 1.8.7

6. Switch to the default Ruby version 1.8.7

7.Download Canvas code

8.Install nginx with Passenger rails

9. Install nginx with passenger rails integrated
passenger-install-nginx-module –prefix=/etc/nginx –auto

Note:- Choose auto download and configure Nginx option and continue

useradd nginx
chown nginx /etc/nginx/ -R

10. Configure Nginx

The passenger-install-nginx-module command above creates an nginx.conf file in /opt/nginx/conf/nginx.conf with directives to enable Phusion Passenger. We will replace this with our own nginx.conf file and configuration structure

Create a new /etc/nginx/conf/nginx.conf file with the following contents:
Replace the passenger version in the below with the installed version. In this case it is passenger-3.0.0.19

create nginx init file to start and stop the services

11. Create database in mysql

12. Canvas default configurations

13. Configure Database details

14. Configure Outgoing email

15. URL configuration for canvas

16. Populate the Database
we need to populate the database with tables and initial data

17. Generate the canvas files

18. create directories and change the file permissions

19. Automated Jobs
Canvas has some automated jobs that need to run at occasional intervals, such as email reports, statistics gathering, and a few other things. Canvas installation will not function properly without support for automated jobs.

20. Start the server

== Virtualization with OpenVZ on CentOS 6 ==

OpenVZ is container-based virtualization for Linux. OpenVZ creates multiple secure, isolated Linux containers on a single physical server enabling better server utilization and ensuring that applications do not conflict. Each container performs and executes exactly like a stand-alone server; a container can be rebooted independently and have root access, users, IP addresses, memory, processes, files, applications, system libraries and configuration files.

1. Pre-Requisites for installing openvz server

2. Add the OpenVZ yum repository

3. Ensure the yum repo points to RHEL6 packages

4. Install the OpenVZ kernel and ensure it’s the 1st option in grub

5. Install the standard OpenVZ utilities

6. Modify /etc/sysctl.conf. Add the below content

7. Create a vmbr0 bridge and add the host’s interface to it

8. Create /etc/vz/vznet.conf with the following content. This will automatically add/remove the container’s interface to the bridge when you start/stop the container.

9. Reboot the machine and boot with openvz kernel

10. Create the openvz container with openvz templates

11. Configure the CT

recode utf8:html < monitor.xml

By Default Ubuntu uses Dash for /bin/sh for performance issues. Though we can execute below command in terminal to change it so that bash is used for /bin/sh instead of Dash. This would expect the user to provide dash or bash to be used for /bin/sh. we need to say <no> to enable bash, instead of dash.

sudo dpkg-reconfigure dash