Sort -u Large Files

on Jan. 31, 2019, 3:06 p.m.

I know this is a huge issue the world is dying to know how to solve. So why not write about it here. Recently, I have acquired several large text files that have... words in them. Here-in lies the problem. For this to be usable, I have to get it down to a single text file with only a set of unique... words. So, after many failed attempts, I found a solution. First, you have the split the large file out into smaller text files using the hashcat utils. Within these utils is a command called splitlen. Use it to split the 600+ GB file out into smaller files. This command splits the files out according to line length.

./splitlen.bin outdir < infile

The above command will output into "outdir" a list of 0-64 files. From there, run the following on the 64 files.

for i in {01..64}; do sort -u -T stmp/ --parallel=8 $i > ${i}.txt ; done;
then
for i in {01..64}; do cat ${i}.txt >> sorted_dedupped.txt ; done;

Now, if you're still interested, let me explain what this does. From my understanding, this does an external merge sort which means it creates temp files to not eat all your RAM to use to then sort these things, slowly reducing till it has your end result file. The -T tells it the location to store the temp files and --parallel tells it to run 8 of these at once(one per CPU code is what I'm doing). Without doing this, sort will just crash. Once you have the 00-64 files sorted, you can combine them altogether and have a unique list.

 

Kali Linux Installing Powershell Core

on June 29, 2017, 7:13 p.m.

sudo apt-get install libunwind8
wget http://security.ubuntu.com/ubuntu/pool/main/i/icu/libicu55_55.1-7ubuntu0.2_amd64.deb
sudo dpkg- i libicu55_55.1-7ubuntu0.2_amd64.deb
wget http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
sudo dpkg -i libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
wget https://github.com/PowerShell/PowerShell/releases/download/v6.0.0-beta.3/powershell_6.0.0-beta.3-1ubuntu1.16.04.1_amd64.deb
sudo dpkg -i powershell_6.0.0-beta.3-1ubuntu1.16.04.1_amd64.deb
powershell


PowerShell v6.0.0-beta.3
Copyright (C) Microsoft Corporation. All rights reserved.

PS /root>

 

Installing PIP on Cygwin

on Nov. 15, 2016, 1:28 p.m.

pip3 Installation, Please

To install pip3, the Python 3-specific version of pip, under Cygwin:

$ python3 -m ensurepip
This assumes the python3 Cygwin package to have been installed, of course.

pip2 Installation, Please

To install both pip and pip2, the Python 2-specific versions of pip, under Cygwin:

$ python -m ensurepip
This assumes the python Cygwin package to have been installed, of course.

 

Weather Station Project

on Sept. 7, 2016, 10:52 p.m.

I was surfing Amazon one day and saw this weather station kit that was based on arduino. There is also a temperture sensor and humidity sensor as well. The problem with kit is it did not allow for the raspberry pi that runs the kit to be put outdoors with the systems. So I built my own case and stand for the station.


mounted weather station

The finished and mounted weather station.




Complete Test fit

This was the complete test fit of all the pieces.




Final component testing

This was the final component testing before I closed up the unit.



 

Installing Masscan

on Jan. 12, 2016, 1:22 p.m.

# Centos
sudo yum groupinstall -y "Development Tools"
sudo yum install -y libpcap libpcap-devel
git clone [email protected]:robertdavidgraham/masscan.git
cd masscan
make
sudo make install
# Ubuntu
sudo apt-get -y install git gcc make libpcap-dev
git clone https://github.com/robertdavidgraham/masscan
cd masscan/
make
sudo make install

 

Logitech c920 Installation

on Oct. 12, 2015, 7:20 p.m.

For some reason, out of the box the Logitech c920 has choppy video but to fix that on Ubuntu 15.04, run the following:

sudo apt-get install dov4l dv4l libv4l-0 libv4l-dev libv4lconvert0 libvideo-capture-v4l-perl qv4l2 v4l-conf v4l-utils v4l2loopback-dkms v4l2loopback-source v4l2ucp

 

Disable ICMP Redirects

on Aug. 6, 2015, 5:05 p.m.


echo 'net.ipv4.conf.all.send_redirects=0' >> /etc/sysctl.conf
echo 'net.ipv4.conf.all.secure_redirects=0' >> /etc/sysctl.conf
echo 'net.ipv4.conf.all.accept_redirects=0' >> /etc/sysctl.conf
echo 'net.ipv6.conf.all.accept_redirects=0' >> /etc/sysctl.conf
sysctl -p

 

Magic Mirror Project

on July 22, 2015, 3:44 p.m.

After seeing this guy's magic mirror, I decided to make my own. There are a few things I did differently. First, I did not use php; strictly css, html, and jquery. Next, I did not spend $100 on a one-way piece of mirror glass. Instead. I bought a sheet of lexan and some one-way self adhesive film. Next, I bought a raspberry pi with most of the stuff I needed. I also bought a 1" by 4" and some molding. There's still some stuff left to be done like sanding, wood-filler(i'm a crap carpenter), and I need to finish securing all the stuff into the frame. Once, I had all the materials, I started to strip down this monitor.


Starting Monitor

I started out with a 23" LCD monitor that i had laying around. It wasn't too hard to strip apart.


Striped Monitor

After striping the monitor down, you will have something like this.


Raspberry Pi

Of course, you have to have the Raspberry Pi in a case and plugged in.



Mirror and Tools

The initial assembly...


Boot-up

The Raspberry Pi booting up...


Beer time

Hey! Looky there! It's way past beer time!



Mirror Time

Up-close of the current date/time. At the top, there is also the sun-up and sun-down times. At the bottom, we have the up-coming holidays.


Mirror Temp

Up-close of the current temperature outside. Up top, is the current wind speed and "feels-like" temperature.


Mirror Moon

Up-close of the bottom. This includes the current moon phase, barometric pressure (whether it's rising or falling), current percentage of humidity, visibility in miles, and then the times of high-tide and low-tide for Tybee Beach.


 

Centos 7 GlusterFS Setup

on June 30, 2015, 6:10 p.m.

A little about the setup before I begin with the actual setup. I created 3 Centos 7 servers with 1 20gb hard disk and 1 200gb hard disk. Since these 3 machines are virtual, it just seemed easier to do this to keep the data separate from the OS disk. Then I installed Centos and set a static IP for each. The following set of commands needs to be run on all 3 servers.

Install the gluster repo:
curl http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo -o /etc/yum.repos.d/glusterfs-epel.repo

Install the epel repo:
yum install epel-release

Delete repo cache:
yum clean all

Install the glusterfs server
yum -y install glusterfs-server

Create partition on data disk:
fdisk /dev/vdb
n
p

Extend the main volume group to cover the new disk
vgextend centos /dev/vdb1

Add logical volume to volume group named data and assign it all the space available
lvcreate -l100%FREE -n data centos

Format the new volume group
mkfs.ext4 /dev/mapper/centos-data

Add firewall rules to allow the 3 servers to talk
firewall-cmd --permanent --zone=trusted --add-source=10.10.10.10/32
firewall-cmd --permanent --zone=trusted --add-source=10.10.10.11/32
firewall-cmd --permanent --zone=trusted --add-source=10.10.10.12/32
firewall-cmd --reload

Start and enable the gluster server
systemctl start glusterd
systemctl enable glusterd

Make directory for gluster
mkdir -p /data/glusterfs

Add host entries so each node knows where the other is
echo "10.10.10.11 gfs01" >> /etc/hosts
echo "10.10.10.12 gfs02" >> /etc/hosts
echo "10.10.10.13 gfs03" >> /etc/hosts

Probe each server to make sure it responds
gluster peer probe gfs01
gluster peer probe gfs02
gluster peer probe gfs03

Check peer status
gluster peer status

Create the glusterfs volume to share. The following command creates 3 copies of all data so each node has a copy of the file
gluster volume create data replica 3 \
gfs01:/data/glusterfs \
gfs02:/data/glusterfs \
gfs03:/data/glusterfs

Start the data volume
gluster volume start data

Check the glusterfs status
gluster volume info

The following is for the client install and mounting of the share:

curl http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo -o /etc/yum.repos.d/glusterfs-epel.repo
yum -y install glusterfs glusterfs-fuse
mount -t glusterfs gfs03.pingnattack.com:/data1 /mnt
df -hT

 

Bastion Host Setup

on April 27, 2015, 4:35 p.m.

This setup is assuming you have a default minimal server install of Centos 7. First, update the system:
yum -y update
Next, install epel repo:
yum install -y epel-release
Next, install fail2ban to stop the script kiddies:
yum install -y fail2ban
Next, install the tools to configure selinux:
yum install policycoreutils-python
Next, we need to setup the selinux policies to allow fail2ban to write syslogs:
vi fail2ban-syslog.te


module fail2ban-syslog 1.0;
require {
type syslogd_var_run_t;
type fail2ban_t;
class dir read;
class file read;
class file open;
class file getattr;
}

#============= fail2ban_t ==============
allow fail2ban_t syslogd_var_run_t:dir read;
allow fail2ban_t syslogd_var_run_t:file read;
allow fail2ban_t syslogd_var_run_t:file open;
allow fail2ban_t syslogd_var_run_t:file getattr;


checkmodule -M -m -o fail2ban-syslog.mod fail2ban-syslog.te
semodule_package -o fail2ban-syslog.pp -m fail2ban-syslog.mod
semodule -i fail2ban-syslog.pp

Next, setup the selinux policy to allow log rotating:
vi fail2ban-logrotate.te


module logrotate-fail2ban 1.7;
require {
type fail2ban_client_exec_t;
type logrotate_t;
type init_var_lib_t;
class file { open read execute getattr write create execute_no_trans setattr unlink ioctl rename};
}

#============= logrotate_t ==============
allow logrotate_t fail2ban_client_exec_t:file execute_no_trans;
allow logrotate_t fail2ban_client_exec_t:file { open read execute ioctl };
allow logrotate_t init_var_lib_t:file { open read getattr write create unlink setattr rename };


checkmodule -M -m -o fail2ban-logrotate.mod fail2ban-logrotate.te
semodule_package -o fail2ban-logrotate.pp -m fail2ban-logrotate.mod
semodule -i fail2ban-logrotate.pp

Next, setup fail2ban configuration:
vi /etc/fail2ban/jail.d/sshd.local


[sshd]
enabled = true
#action = firewallcmd-ipset
bantime = 86400

Next, enable fail2ban on startup and start the service:
systemctl enable fail2ban
systemctl start fail2ban

Next, let's harden the ssh server a bit. First, let's limit logins to ssh key authentication only:
vi /etc/ssh/sshd_config


...
PermitRootLogin without-password
...
PasswordAuthentication no
...

Most people still lockout direct login to root but honestly this is a mentality left over from telnet days.

Next, limit your Kex, Ciphers, and MACs. Ideally, you would want to do the following but this is only really going to work if you use linux and not a Macbook or Putty:


KexAlgorithms [email protected],diffie-hellman-group-exchange-sha256
Ciphers [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr
MACs [email protected],[email protected],[email protected],[email protected],hmac-sha2-512,hmac-sha2-256,hmac-ripemd160,[email protected]

But more than likely, this will work a little better for you:


KexAlgorithms diffie-hellman-group-exchange-sha256
MACs [email protected],[email protected],[email protected],[email protected],hmac-sha2-512,hmac-sha2-256,hmac-ripemd160,[email protected]
Ciphers aes256-ctr,aes192-ctr,aes128-ctr

After thought; maybe you mistakingly got yourself blocked. Here is how to remove a block. First issue this command to figure out the IP blocked:
fail2ban-client status sshd
fail2ban-client set sshd unbanip 192.168.1.100

 

FreeIPA Client Install on Centos 7

on Feb. 12, 2015, 10:47 a.m.

First thing to check is that you have your first DNS server pointer to your IPA server

Next, make sure you have your fqdn in your host file
vi /etc/hosts


127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.5 client.example.com client

Set home directories to be auto created
yum -y install ipa-client pam_mkhomedir.so oddjob-mkhomedir
echo "session required pam_mkhomedir.so skel=/etc/skel/ umask=0022" >> /etc/pam.d/system-auth
authconfig --enablemkhomedir --update

Tell the nsswitch to look at ldap for sudoers
echo sudoers: files ldap >> /etc/nsswitch.conf

If ubuntu add sudo to /etc/sssd/sssd.conf
services = nss, pam, ssh, sudo

Add the bind account info


cat << EOF > /etc/sudo-ldap.conf
binddn uid=sudo,cn=sysaccounts,cn=etc,dc=example,dc=com
bindpw
ssl start_tls
tls_cacertfile /etc/ipa/ca.crt
tls_checkpeer yes
uri ldap://ipa.example.com
sudoers_base ou=SUDOers,dc=example,dc=com
EOF

Set the nis lookup domain


cat << EOF >> /etc/rc.d/rc.local
nisdomainname example.com
EOF

Run the install for the client. It should have all the settings predefined for you if you have DNS and hostname setup.
ipa-client-install

Enable necessary services to start on boot
systemctl enable oddjobd
systemctl enable sssd

 

Freeipa Server Setup on Centos 7

on Feb. 12, 2015, 6:37 a.m.

The second part of my homelab redo was to setup a Freeipa server.

First thing to check is that you have your fqdn in your host file

vi /etc/hosts


127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.5 ipa.example.com ipa

Install the necessary packages
yum -y install ipa-server bind-dyndb-ldap

Issue this command to start the configuration of IPA
ipa-server-install --setup-dns

Add the necessary rules to firewalld
firewall-cmd --permanent --add-service=ntp
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --permanent --add-service=ldap
firewall-cmd --permanent --add-service=ldaps
firewall-cmd --permanent --add-service=kerberos
firewall-cmd --permanent --add-service=kpasswd
firewall-cmd --permanent --add-service=dns
firewall-cmd --reload

Set home directories to be auto created
yum -y install ipa-client pam_mkhomedir.so oddjob-mkhomedir
echo "session required pam_mkhomedir.so skel=/etc/skel/ umask=0022" >> /etc/pam.d/system-auth
authconfig --enablemkhomedir --update

Create sudo lookup account to bind too.
ldappasswd -x -S -W -h ipa.example.com -ZZ -D "cn=Directory Manager" uid=sudo,cn=sysaccounts,cn=etc,dc=example,dc=com

Tell the nsswitch to look at ldap for sudoers
echo sudoers: files ldap >> /etc/nsswitch.conf

If ubuntu add sudo to /etc/sssd/sssd.conf
services = nss, pam, ssh, sudo

Add the bind account info


cat << EOF > /etc/sudo-ldap.conf
binddn uid=sudo,cn=sysaccounts,cn=etc,dc=example,dc=com
bindpw
ssl start_tls
tls_cacertfile /etc/ipa/ca.crt
tls_checkpeer yes
uri ldap://ipa.example.com
sudoers_base ou=SUDOers,dc=example,dc=com
EOF

Set the nis lookup domain


cat << EOF >> /etc/rc.d/rc.local
nisdomainname example.com
EOF

Enable and start necessary services
systemctl enable oddjobd
systemctl start oddjobd
systemctl enable sssd
systemctl start sssd

 

Centos 7 and KVM

on Feb. 12, 2015, 6:12 a.m.

So recently, I have started redoing my homelab in Centos 7. First step was to redo my hypervisor in Centos 7 with KVM.

First, install the necessary packages.
yum -y install kvm virt-manager libvirt virt-install qemu-kvm xauth dejavu-lgc-sans-fonts bridge-utils policycoreutils-python

Enable the kernel to do IP forwarding
echo "net.ipv4.ip_forward = 1"|sudo tee /etc/sysctl.d/99-ipforward.conf

Apply the new kernel settings
sysctl -p /etc/sysctl.d/99-ipforward.conf

Start libvirt and set it to start on boot
systemctl start libvirtd
systemctl enable libvirtd

Create a network bridge for the VM's to share
vi /etc/sysconfig/network-scripts/ifcfg-br0


DEVICE="br0"
ONBOOT="yes"
TYPE="Bridge"
BOOTPROTO="dhcp"
STP="on"
DELAY="0.0"

Tell your network interface to be a slave to the bridge
vi /etc/sysconfig/network-scripts/ifcfg-enp6s0f0


DEVICE="enp3s0"
ONBOOT="yes"
BRIDGE="br0"

Make sure to set your hostname
vi /etc/hostname


hypervisor01.example.com

Start networking to make sure the changes take effect
systemctl restart network

Make the directory to be your VM image folder
mkdir /data

Set the selinux policy on that new folder
semanage fcontext -a -t virt_image_t "/data(/.*)?"
restorecon -R /data

Tell libvirt you want to use the above folder for VMs
virsh pool-destroy default
virsh pool-undefine default
virsh pool-define-as --name default --type dir --target /data
virsh pool-autostart default
virsh pool-build default
virsh pool-start default

 

Universal Profile

on Feb. 2, 2015, 2:12 p.m.


# HP-UX
if [ `uname` == HP-UX ]; then
export PS1="$(whoami)@$(hostname):"'$PWD'"$ "
fi

# AIX
if [ `uname` == AIX ]; then
export PS1="$(whoami)@$(hostname):"'$PWD'"$ "
fi

# Solaris
if [ `uname` == SunOS ]; then
export PS1="[email protected]$(hostname):"'$PWD'"$ "
if [ -e /bin/bash ]; then
export SHELL=/usr/bin/bash
export PS1="\[email protected]\h:\w$ "
exec $SHELL
fi
fi

# Linux
if [ `uname` == Linux ]; then
export PS1="\[email protected]\h:\w$ "
fi

 

Set Primary Group in AIX

on Feb. 2, 2015, 12:39 p.m.

chuser pgrp=staff username

 

Allowing MTR through your Cisco ASA

on Jan. 6, 2015, 4:33 p.m.

Here's how I was able to get the wonderful tool mtr to go through my firewall:

asa-gw(config)# class-map icmp-class
asa-gw(config-cmap)# match default-inspection-traffic
asa-gw(config-cmap)# exit
asa-gw(config)# policy-map icmp_policy
asa-gw(config-pmap)# class icmp-class
asa-gw(config-pmap-c)# inspect icmp
asa-gw(config-pmap-c)# exit
asa-gw(config-pmap)# exit
asa-gw(config)# service-policy icmp_policy interface outside

 

Java 8u25 Install on Centos 6.6

on Jan. 2, 2015, 6:38 p.m.

here's how:

cd /opt/

wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u25-b17/jdk-8u25-linux-x64.tar.gz"

tar xzf jdk-8u25-linux-x64.tar.gz
alternatives --install /usr/bin/java java /opt/jdk1.8.0_25/bin/java 2
alternatives --config java
alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_25/bin/jar 2
alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_25/bin/javac 2
alternatives --set jar /opt/jdk1.8.0_25/bin/jar
alternatives --set javac /opt/jdk1.8.0_25/bin/javac
cd /opt/jdk1.8.0_25/jre
export JAVA_HOME=/opt/jdk1.8.0_25
export JRE_HOME=/opt/jdk1.8.0_25/jre
export PATH=$PATH:/opt/jdk1.8.0_25/bin:/opt/jdk1.8.0_25/jre/bin

 

Speed Up Youtube Videos

on Nov. 22, 2014, 7:10 p.m.

for linux:
sudo iptables -A INPUT -s 173.194.55.0/24 -j REJECT
sudo iptables -A INPUT -s 206.111.0.0/16 -j REJECT

for mac:
sudo ipfw add reject src-ip 173.194.55.0/24 in
sudo ipfw add reject src-ip 206.111.0.0/16 in

for windows:

netsh advfirewall firewall add rule name="YouTubePerformanceHack" dir=in
action=block remoteip=173.194.55.0/24,206.111.0.0/16 enable=yes

 

Disk Space Analysis

on Nov. 22, 2014, 7:07 p.m.

du -sk * | sort -rn | more

 

Red Hat Enterprise Linux Reset Root Password

on Nov. 22, 2014, 7:06 p.m.

On rhel, this is how you reset the root password. Reboot the server
and at the grub boot menu, press tab. This is then give you the option
to edit the current grub config. Press e then highlight the kernel
line and press e again. Add a space to the end of the line and add

init=/bin/bash

Once that is done, press enter and then press b to boot. The
system will now boot up into a read-only single user bash shell.
Once at the shell, run

mount -o remount,rw /

This will remount the filesystem with write permissions. Now run

passwd root

This will allow you to reset the root password. Afterwards, reboot. :)

 

Solaris Console Escape Sequence

on Nov. 22, 2014, 6:06 p.m.

Since I never can remember the escape sequence for the Solaris serial console, I am going to put it here
#.

 

Solaris Profile

on Aug. 31, 2014, 1:06 p.m.

Today I have been screwing around with my Solaris box. Now I will admit I prefer Linux to Solaris so I have been making an effort to at least get my bash shell straightened out. To do this, I did the following:

vi /etc/passwd
...
root:x:0:0:Super-User:/root:/usr/bin/bash
...

vi /root/.profile

export PS1="\[email protected]\h:\w # "
export TERM=xterm
export SHELL=/usr/bin/bash
exec $SHELL

 

NIS and Linux

on July 9, 2014, 6:09 a.m.

Remember to set the domainname for nis in /etc/defaultdomain or else ypbind will fail to bind if your machine domain.

echo 'yourdomain' > /etc/defaultdomain
chkconfig rpcbind on
chkconfig ypbind on
domainname yourdomain
service ypbind start
ypcat passwd | less

 

Another Reason to Use Visudo

on Jan. 21, 2014, 4:07 p.m.

Today I learned that one REALLY should use visudo. Today a co-worker was on a Solaris box after a while called me over because he could not figure out how this user group had sudo rights when their was no mention of it in /etc/sudoers. After both being stumped for about 10 minutes, I asked if the user id of the user was id 0; negative. I paused then thought "what happens when you run visudo?" Sure enough there it was but it lead to the question of "where the hell was the configuration file for the groups?!" After some googling, my coworker found a small article that said:


Neither Solaris 9 or 10 include sudo - it wasn't bundled with Solaris
until Solaris 11 - so for Solaris 9 & 10 the answer is “Whatever path
was compiled into whatever version you installed.” However, one of the
places to get sudo is OpenCSW. You can install a precompiled package.
The sudoers file is then /etc/opt/csw/sudoers.

So there you go, /etc/opt/csw/sudoers, but in the future just use visudo.

 

Measuring Server AMPs

on Dec. 20, 2013, 5:15 p.m.

So today I learned how to measure amps on a server. First to do so requires some prep work.




We start by striping the insulation off a power cable since a clamp meter is only designed to be clamped around either the positive or negative wire but not both at the same time.



This should be enough. Next let us power up our server and see how many amps 1 side of the twin 1u supermicro server pulls.



1.8 amps? How have I not tripped a breaker?! Oh wait, I have... Next let us power up side 2.





Looks like we are pulling 3.4 amps at power up. Next let us get a reading at idle.



2.7amps it looks like. Next let us see what I pulls under load.



2.8amps-3.0amps it seems with both sides calculating Pi out to the 20th digit.

 

Reset AIX Account

on Dec. 19, 2013, 8:55 a.m.

This is how you reset the failed login count on AIX
chsec -f /etc/security/lastlog -a "unsuccessful_login_count=0" -s username

This is how you unlock the account
chuser "account_locked=false" username

 

Centos: Install package using yum

on Dec. 5, 2013, 5:31 a.m.

yum --nogpgcheck localinstall packagename.arch.rpm

 

Sed - remove commented lines and blank lines

on Nov. 24, 2013, 9:59 a.m.

Because i keep forgetting this sweet little one liner, I am putting it here.
sed -e 's/#.*$//' -e '/^$/d' inputFile

 

Disable Selinux

on Nov. 22, 2013, 2:04 p.m.

echo 0 > /selinux/enforce
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

 

Fixing Time Zones

on Nov. 21, 2013, 9:25 a.m.

This is how to change your system time to the correct time zone on redhat/centos
mv /etc/localtime /etc/localtime.old
ln -s /usr/share/zoneinfo/America/New_York /etc/localtime

 

Clearing Password History

on Nov. 19, 2013, 6:05 a.m.

Occasionally one has to violate best practice to clear password history... here is how to do it on

AIX, and Linux:
chuser histsize=0 username
Sometimes on AIX you have to make sure these lines exist too.
edit this file so that root has a minage and histexpire:
vi /etc/security/user
minage=0
histexpire=0

Solaris
chmod 600 /etc/security/passhistory
vi /etc/security/passhistory (delete the line for your user here)
chmod 400 /etc/security/passhistory

 

VLC and Blurays

on Oct. 21, 2013, 9:35 a.m.

Ever decided to no longer use Windows but still wanted to use your blu-ray drive to play your movie? yeah, me too.. heres how to set it up to run with VLC.

Code:
sudo add-apt-repository ppa:n-muench/vlc
sudo apt-get update
sudo apt-get install vlc libaacs0 libbluray-bdj libbluray1
sudo apt-get dist-upgrade
cd ~/
mkdir -p ~/.config/aacs/
cd ~/.config/aacs/ && wget http://vlc-bluray.whoknowsmy.name/files/KEYDB.cfg

 

Add User to Sudo Group

on Oct. 4, 2013, 6:42 a.m.

Because I keep forgetting how to do this, this is how one adds an existing user to the wheel group
usermod -a -G group user

 

Ubuntu IPtables Configuration

on Oct. 3, 2013, 7:58 p.m.

This is the init.d script that's needed since ubuntu doesn't come with one.


#!/bin/bash
IPT=/sbin/iptables
IPT6=/sbin/ip6tables
IPTR=/sbin/iptables-restore
IPTCONFIG=/root/.config/fw/firewall.conf
PROG=iptables

_stop(){
### reset ipv4 iptales ###
echo "Stopping iptables..."
$IPT -F
$IPT -X
$IPT -Z
for table in $( do
$IPT -t $table -F
$IPT -t $table -X
$IPT -t $table -Z
done
$IPT -P INPUT ACCEPT
$IPT -P OUTPUT ACCEPT
$IPT -P FORWARD ACCEPT
### reset ipv6 iptales ###
$IPT6 -F
$IPT6 -X
$IPT6 -Z
for table in $( do
$IPT6 -t $table -F
$IPT6 -t $table -X
$IPT6 -t $table -Z
done
$IPT6 -P INPUT ACCEPT
$IPT6 -P OUTPUT ACCEPT
$IPT6 -P FORWARD ACCEPT

}

_start(){
### add your commands here ###
echo "Starting iptables..."
$IPTR < "$IPTCONFIG"
}

# See how we were called.
case "$1" in
start)
_start
;;
stop)
_stop
;;
restart)
_stop
_start
;;
*)
echo $"Usage: $PROG {start|stop|restart}"
exit 1
esac

This is how you save it
mkdir -p /root/.config/fw/
iptables-save > /root/.config/fw/firewall.conf

 

Selinux, PHP, and HTTPD

on Oct. 1, 2013, 7:07 a.m.

A lot of tutorials say to disable selinux. This is dumb. here is how you configure it to let you serve php files.

first make sure it is actually selinux
echo 0 > /selinux/enforce
This disables selinux so now let us see if the pages serve up now.
Next if the pages serve fine with selinux disable, see what the policy is currently set on the directory
ls -Z
Also, you can check here to see if it is selinux that is keeping apache from executing the php files
tail /var/log/audit/audit.log
Next, we need to install some things to configure selinux
yum -y install policycoreutils-python
Next, lets configure it to work with the default directory
semanage fcontext -a -t httpd_sys_script_exec_t '/var/www/html/(/.*)?'
restorecon -R -v /var/www/html/
See if it changed the polcy
ls -Z
Re-enable selinux
echo 1 > /selinux/enforce

 

Xubuntu Disable Network Manager

on Sept. 30, 2013, 4:39 p.m.

this disables it:
sudo touch /etc/default/NetworkManager
sudo touch /etc/default/NetworkManagerDispatcher
echo "exit" | sudo tee -a /etc/default/NetworkManager
echo "exit" | sudo tee -a /etc/default/NetworkManagerDispatcher

and this re-enables it:
sudo rm /etc/default/NetworkManager
sudo rm /etc/default/NetworkManagerDispatcher

 

Development Tools Uninstall

on Sept. 15, 2013, 7:42 a.m.

Yes, initially one can do the following but i find it removes things I find useful on my boxes.. like ssh-client and ssh server.
yum groupremove "Development Tools"
So that is why I do this; Should work great on x64 systems.

yum erase autoconf.noarch automake.noarch bison.x86_64 byacc.x86_64 cscope.x86_64 ctags.x86_64 cvs.x86_64 diffstat.x86_64 doxygen.x86_64 elfutils.x86_64 flex.x86_64 gcc.x86_64 gcc-c++.x86_64 gcc-gfortran.x86_64 gettext.x86_64 git.x86_64 indent.x86_64 intltool.noarch libtool.x86_64 make.x86_64 patch.x86_64 patchutils.x86_64 pkgconfig.x86_64 rcs.x86_64 redhat-rpm-config.noarch rpm-build.x86_64 subversion.x86_64 swig.x86_64 systemtap.x86_64 alsa-lib.x86_64 apr.x86_64 apr-util.x86_64 atk.x86_64 avahi-libs.x86_64 cairo.x86_64 cloog-ppl.x86_64 cpp.x86_64 cups-libs.x86_64 elfutils-libs.x86_64 fontconfig.x86_64 freetype.x86_64 gdb.x86_64 gettext-devel.x86_64 gettext-libs.x86_64 glibc-devel.x86_64 glibc-headers.x86_64 gnutls.x86_64 gtk2.x86_64 hicolor-icon-theme.noarch jasper-libs.x86_64 kernel-devel.x86_64 kernel-headers.x86_64 libICE.x86_64 libSM.x86_64 libX11.x86_64 libX11-common.noarch libXau.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXfixes.x86_64 libXft.x86_64 libXi.x86_64 libXinerama.x86_64 libXrandr.x86_64 libXrender.x86_64 libXtst.x86_64 libart_lgpl.x86_64 libgcj.x86_64 libgfortran.x86_64 libgomp.x86_64 libjpeg-turbo.x86_64 libpng.x86_64 libproxy.x86_64 libproxy-bin.x86_64 libproxy-python.x86_64 libstdc++-devel.x86_64 libtasn1.x86_64 libthai.x86_64 libtiff.x86_64 libxcb.x86_64 mailcap.noarch mpfr.x86_64 neon.x86_64 pakchois.x86_64 pango.x86_64 perl.x86_64 perl-Compress-Raw-Zlib.x86_64 perl-Compress-Zlib.x86_64 perl-Error.noarch perl-Git.noarch perl-HTML-Parser.x86_64 perl-HTML-Tagset.noarch perl-IO-Compress-Base.x86_64 perl-IO-Compress-Zlib.x86_64 perl-Module-Pluggable.x86_64 perl-Pod-Escapes.x86_64 perl-Pod-Simple.x86_64 perl-URI.noarch perl-XML-Parser.x86_64 perl-libs.x86_64 perl-libwww-perl.noarch perl-version.x86_64 pixman.x86_64 ppl.x86_64 rsync.x86_64 systemtap-client.x86_64 systemtap-devel.x86_64 systemtap-runtime.x86_64 unzip.x86_64 xz.x86_64 xz-lzma-compat.x86_64 zip.x86_64

 

Installing Syslog-ng on Centos 6.4

on Aug. 2, 2013, 10:37 a.m.

First you will need to enable the ELEP repo
wget https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum update

next install syslog-ng
yum install syslog-ng

next if we are using a mysql backend, we wil have to install the plugin and mysql server
yum install syslog-ng-libdbi mysql-server

 

Yubikey + SSH key Authenication

on July 22, 2013, 4:31 p.m.

So at work we use Yubikeys for various things. So i finally decided to spend the 30 bucks and get one myself for Lastpass and authentication for my servers. While i already have ssh key only authenication setup for most all of my servers, i figured why not for giggles setup up my yubikey as well.

So setting up two factor authentication for ssh isnt as hard as it sounds. I will now walk one through how to do this on Centos 6.4.

First you will need to enable the ELEP repo
wget https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum update

Next you will need to install the Yubikey pam module
yum install pam_yubico

Next open /etc/pam.d/sshd and add this line under #%PAM-1.0
nano /etc/pam.d/sshd
line to be added:
auth required pam_yubico.so id=16 authfile=/etc/yubikey_mappings

Next, you will need to map the yubikey output to individual users
nano /etc/yubikey_mappings
add lines like this where aaaaaaaaaaaa is the first 12 charactor output of your Yubikey
root:aaaaaaaaaaaa

Next, add this to the bottom of your /etc/ssh/sshd_config file
nano /etc/ssh/sshd_config
line to be added:
RequiredAuthentications2 publickey,password

Finally, restart the ssh service
service sshd restart

 

Top Replacement: HTop

on July 6, 2013, 6:44 p.m.

I also ran across a top replacement today
sudo apt-get install htop

 

vnStat

on July 6, 2013, 6:38 p.m.

Today I came across a cool tool for monitoring network usage in real time as well as historical on a linux called vnStat
sudo apt-get install vnstat
and if you want it to output images
sudo apt-get install vnstati

 

Bind DNS Query Logging

on July 5, 2013, 4:17 p.m.

Turn on logging
rndc querylog

View bind sever query log
tail -f /var/log/syslog

Turn off logging
rndc querylog

 

List Ubuntu Hardware

on July 4, 2013, 6:28 p.m.

Ever needed to find out what hardware a system was running out in more detail that just running
cat /proc/cpuinfo
and
cat /proc/meminfo
If so, run the following...
lshw

 

Xubuntu 12.04 Login Fix

on June 13, 2013, 5:37 p.m.

Xubuntu 12.04 has a bug which slows down the login process dramatically. Luckily, a fix is available:
sudo add-apt-repository ppa:andreas-diesner/lightdm-fix-temporary
sudo apt-get update
sudo apt-get upgrade

 

Fixing My Slow SSH Login

on June 2, 2013, 7:39 a.m.

To disable GSS API, first open/etc/ssh/sshd_config in your favorite editor

vi /etc/ssh/sshd_config

Then find the line that says GSSAPIAuthentication yes and change it to GSSAPIAuthentication no

sed -i 's/GSSAPIAuthentication yes/GSSAPIAuthentication no/g' etc/ssh/sshd_config

and save the file. Finally, restart the sshd service.

service sshd restart

 

Identifying failed drive in FreeNas

on April 24, 2013, 9:54 a.m.

[[email protected]] ~# zpool status -v
pool: data
state: DEGRADED
status: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the entire pool from backup.
see: http://www.sun.com/msg/ZFS-8000-8A
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
data DEGRADED 0 0 18
raidz1 DEGRADED 0 0 50
ada0p2 ONLINE 0 0 4
ada1p2 ONLINE 0 0 0
3024388728382954071 UNAVAIL 0 0 0 was /dev/ada1p2
ada2p2 ONLINE 0 0 0
ada3p2 ONLINE 0 0 0


[[email protected]] /data# sqlite3 freenas-v1.db
SQLite version 3.7.9 2011-11-01 00:52:41
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from storage_disk;
1|Disabled|Always On|S24087AK||{serial}S24087AK|1|Disabled|Auto||||2|ada0
1|Disabled|Always On|S24087NL||{serial}S24087NL|1|Disabled|Auto||||4|ada1
1|Disabled|Always On|S2408AAT||{serial}S2408AAT|1|Disabled|Auto||||5|ada2
1|Disabled|Always On|S2408AGL||{serial}S2408AGL|1|Disabled|Auto||||6|ada3
0|Disabled|Always On|||{devicename}da0|1|Disabled|Auto||||7|da0

Then look for the drive that has a serial number that is not listed above.

 

Init.d script template

on March 26, 2013, 4:58 p.m.

#!/bin/bash
# myapp daemon
# chkconfig: 345 20 80
# description: myapp daemon
# processname: myapp

DAEMON_PATH="/usr/sbin"

DAEMON=myapp
DAEMONOPTS="-my opts"

NAME=myapp
DESC="My daemon description"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

case "$1" in
start)
printf "%-50s" "Starting $NAME..."
cd $DAEMON_PATH
PID=`$DAEMON $DAEMONOPTS > /dev/null 2>&1 & echo $!`
#echo "Saving PID" $PID " to " $PIDFILE
if [ -z $PID ]; then
printf "%s\n" "Fail"
else
echo $PID > $PIDFILE
printf "%s\n" "Ok"
fi
;;
status)
printf "%-50s" "Checking $NAME..."
if [ -f $PIDFILE ]; then
PID=`cat $PIDFILE`
if [ -z "`ps axf | grep ${PID} | grep -v grep`" ]; then
printf "%s\n" "Process dead but pidfile exists"
else
echo "Running"
fi
else
printf "%s\n" "Service not running"
fi
;;
stop)
printf "%-50s" "Stopping $NAME"
PID=`cat $PIDFILE`
cd $DAEMON_PATH
if [ -f $PIDFILE ]; then
kill -HUP $PID
printf "%s\n" "Ok"
rm -f $PIDFILE
else
printf "%s\n" "pidfile not found"
fi
;;

restart)
$0 stop
$0 start
;;

*)
echo "Usage: $0 {status|start|stop|restart}"
exit 1
esac

 

What's Eating My Ram?!

on March 25, 2013, 8:23 a.m.

ps -e -o pid,vsz,comm= | sort -n -k 2

 

Deleting Files Older Than 5 Days

on Feb. 6, 2013, 5:55 p.m.

find /path/to/files* -mtime +5 -exec rm {} \;

 

xenserver url installs or netinstalls

on Feb. 6, 2013, 5:53 p.m.

for xenserver template installs or just netinstalls... use these urls to pull the install.img files

centos:
http://mirror.centos.org/centos/6.2/os/i386
http://mirror.centos.org/centos/6.3/os/x86_64

ubuntu:
http://archive.ubuntu.net/ubuntu/

debian:
http://ftp.nl.debian.org/debian/

 

NIC Renamed eth0 to eth1

on Jan. 29, 2013, 10:58 a.m.

Ever had to replace hardware in a linux box and then the networking wanted to screw with you? as in not find your NIC but lspci would list the NIC?

Yeah, this is the file is causing that issue. It records the MAC address of your NIC and then if it changes, it will rename your primary NIC as eth1. Also, if your new NIC name(eth1) isn't in your /etc/network/interfaces, you won't have any networking.


rm -rf /etc/udev/rules.d/*-persistent-net.rules

once you have ran this command, reboot and your should be good to go.

 

Ubuntu NIC Bonding

on Jan. 19, 2013, 2:03 p.m.

auto bond0
iface bond0 inet static
address 192.168.1.1
netmask 255.255.255.0
gateway 192.168.1.254
dns-nameservers 192.168.1.254
post-up ifenslave bond0 eth0 eth1
pre-down ifenslave -d bond0 eth0 eth1

 

KVM Centos 6 and Libvirt

on Jan. 19, 2013, 1:58 p.m.

To install KVM on Centos 6, we first need to install some things.
yum groupinstall Virtualization "Virtualization Client" "Virtualization Platform" "Virtualization Tools"
yum -y install bridge-utils avahi ntp ntp-doc ntpdate nano wget

Next, we need to enable the the epel just in case. To do this, we need to download the following rpm's.
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Install and prioritize the repo's.
rpm -Uvh epel-release-6*.rpm

Now update everything...
yum update
yum upgrade

restart the things and chkconfig the things on...
/etc/init.d/messagebus restart
/etc/init.d/avahi-daemon restart
/etc/init.d/libvirtd restart
/sbin/chkconfig messagebus on
/sbin/chkconfig avahi-daemon on
/sbin/chkconfig libvirtd on

Next, set a static ip for your box..
nano /etc/sysconfig/network-scripts/ifcfg-eth0


DEVICE=eth0
HWADDR=00:11:22:33:44:55
ONBOOT=yes
BRIDGE=br0
NM_CONTROLLED=no

/etc/sysconfig/network-scripts/ifcfg-br0


DEVICE=br0
TYPE=Bridge
DELAY=0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.20.254.201
NETMASK=255.255.255.0
GATEWAY=10.20.254.1
NETWORK=10.20.254.0
DNS1=8.8.8.8
DNS2=8.8.4.4
PEERDNS=yes
NM_CONTROLLED=no

Next, let us tell iptables to allow kvm guest to have their networking bridged
iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

Next, save and restart iptables
service iptables save
service iptables restart

change your kernel limit...
nano /etc/sysctl.conf


# Improve bridge performance
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

apply the changes..
sysctl -p

create a group to add kvm libvirt users too..
groupadd libvirt
chgrp -R kvm /dev/kvm
usermod -a -G libvirt username
usermod -a -G kvm username

configure selinux to play nice with kvm
/etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla


[libvirt Management Access]
# For allowing access to specific user only:
#Identity=unix-user:bozz
# For allowing access to a group (like this guide):
Identity=unix-group:libvirt
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

set the correct permission on the kvm image dir...
chown root:libvirt /var/lib/libvirt/images
chmod g+rw /var/lib/libvirt/images

Setup Network Time Protocal
chkconfig --levels 235 ntpd on
ntpdate 0.pool.ntp.org
/etc/init.d/ntpd start

Let us get some isos to install vm's from...
cd /var/lib/libvirt/images/
wget http://www.gtlib.gatech.edu/pub/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-minimal.iso
wget http://www.gtlib.gatech.edu/pub/ubuntu-releases/12.04.1/ubuntu-12.04.1-server-i386.iso

NOTE: Sometimes you have to delete the default libvirt nat bridge to get bridging to work...
brctl delbr virbr0