Recoving Arch Linux Failed System Upgrade

Once in a while, it's nice to do a complete system upgrade in Arch Linux. All the packages will be updated to the latest bleeding edge release.

However, if for some reason the package upgrade failed, perhaps due to power outage or something unexpected. In my case, I left my system upgrading over night, and unfortunately, in the morning the screen is all black and nothing works. I tried rebooting, but even GRUB didn't show up!

Upgrading arch system

# Beware this might take a long time to run. Could be a few hours. sudo pacman -Syu

Fixing Corrupted Upgrade

1. Boot to Ubuntu live CD
2. Figure out partition where your Arch is installed
sudo fdisk -l
3. Mount the partition
sudo mount /dev/sdaX /mnt
4. Mount devices
mount -t proc /proc mnt/proc
mount --rbind /sys mnt/sys
mount --rbind /dev mnt/dev
5. Chroot to Arch, this basically is to get into Arch system using the current Ubuntu's kernel
sudo chroot /mnt
6. Now that you're inside of your Arch, you can do whatever needed to fix your installation. Below are some stuffs that might be needed
# Finish unfinished Arch upgrade.
# To see if you need this, check /var/log/pacman.log. There you'll see what happened on your last upgrade. (i.e. upgrade got interrupted in the middle) sudo pacman -Syu

# Re-create RAM disk, this creates device initialization configuration that kernel uses to boot
# For more info: https://wiki.archlinux.org/index.php/mkinitcpio
sudo mkinitcpio -p linux

# Fix GRUB installation. In my case, GRUB didn't show up on boot, so perhaps it was corrupted or something
sudo grub-mkconfig > /boot/grub.cfg # Unlike in normal environment, /boot here is mounted as RW, so write here will persist
# P.S: If you dual boot Arch with other distro, mount the other distro before running grub-mkconfig. If you do so
# GRUB will automatically detect it and create appropriate entry in bootloader

7. Reboot your system! :)
Hopefully this is helpful. Much thanks to Mort Yao's article: https://www.soimort.org/notes/170407/

Fixing Unstable WiFi in Latest Ubuntu/Arch Linux

My laptop is Thinkpad Yoga S3 (2015). I happened to run into an issue where intermittently my Internet connection (via WiFi) wouldn't work, although my other devices worked just fine. When this happened, I re-connected my WiFi and the Internet worked again! Initially I thought this was because of my Ubuntu 16.04 wasn't stable. But after upgrading to 18.04, the problem still persisted! I have finally figured it out and fixed it though :)


1. ThinkPad Yoga S3 has Intel 7265D card, apparently the latest firmware is buggy. So if you run into the same issue, check if your WiFi card the same version

$ sudo lshw -C network

You'll see something like this:
  *-network                
       description: Wireless interface
       product: Wireless 7265
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:02:00.0
       logical name: wlp2s0
       version: 59
       serial: 60:57:18:ad:54:3f
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress bus_master cap_list ethernet physical wireless
       configuration: broadcast=yes driver=iwlwifi driverversion=4.16.10-1-ARCH firmware=22.391740.0 ip=192.168.100.4 latency=0 link=yes multicast=yes wireless=IEEE 802.11
       resources: irq:46 memory:d4100000-d4101fff

2. Check what version of driver is currently running
```
$ dmesg | grep iwlwifi

You'll see something like this:

Dec 20 23:00:08 drax kernel: iwlwifi 0000:03:00.0: Microcode SW error detected.  Restarting 0x2000000.
Dec 20 23:00:08 drax kernel: iwlwifi 0000:03:00.0: Loaded firmware version: 27.168.6.1

```

3. The issue is apparently that the newest firmware is buggy. In my case, version 29 and 27 are both buggy. This explains why back when I was using 14.04, things were nice and stable.

4. To fix it, we have to downgrade the firmware. It's really easy. There is a directory where all different firmware versions for the WiFi card is located. On boot, the system loads the latest version. So to downgrade, we just have to remove the latest version so the system would pick the previous one

```
Check firmwares version that are available:
$ ls /usr/lib/firmware/iwlwifi-7265D-*.ucode

In my case:
lrwxrwxrwx 1 root root      21 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-10.ucode -> iwlwifi-7265-10.ucode
-rw-r--r-- 1 root root 1002800 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-12.ucode
-rw-r--r-- 1 root root 1008692 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-13.ucode
-rw-r--r-- 1 root root 1384500 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-16.ucode
-rw-r--r-- 1 root root 1383604 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-17.ucode
-rw-r--r-- 1 root root 1385368 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-21.ucode
-rw-r--r-- 1 root root 1028376 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-27.ucode
-rw-r--r-- 1 root root 1028376 May  9 00:27 /usr/lib/firmware/iwlwifi-7265D-29.ucode
In my case, versions 29 and 27 are buggy. So I moved it away
$ sudo mv  /usr/lib/firmware/iwlwifi-7265D-27.ucode ~/tmp
$ sudo mv  /usr/lib/firmware/iwlwifi-7265D-29.ucode ~/tmp
```

5. Restart, the system. Things are working fine now :)

Upgrading Ubuntu Linux Kernel

Sometimes there are problems with the kernel that a Ubuntu distro ships with. For example, on 16.04 LTS release image, my ThinkPad Yoga 14 S3 often fails to resume from sleep. And the latest version of this writing, 4.13.12, caused intermittent WiFi issue which required me to re-connect to the AP periodically.

In short, sometimes there are needs to switch to different kernel versions (sometimes older) to find the one that is the most stable for the hardware in subject.

Here's how you upgrade Linux Kernel:
1. Pick the kernel to update to in:
http://kernel.ubuntu.com/~kernel-ppa/mainline/

2. Download the 'generic' version of the right architecture (3 deb files)

3. Install the kernel from terminal
sudo dpkg -i *.deb

4. Reboot your system!

Tips:
You can boot to a different kernel version in Ubuntu. While booting up, once you get into GRUB bootloader, pick "Advanced Option" and select the version of kernel you want to use.

Saving Second Monitor Settings Across Reboots in Ubuntu 16.04

I recently used a second monitor with my Ubuntu 16.04 laptop. After a while, I realized that whenever the HDMI is re-plugged or if the computer is rebooted, the setting reset back!

For example, if I set the monitor to be at the left of my monitor through preferences -> display, it got reset back to be at the right of my laptop after a reboot.

The solution is to use xrandr:
First, run:

#This gives out information about ID of display that is currently running on the system.


xrandr

#After that, run command like the following to update the setting
xrandr --output HDMI2 --left-of eDP1


#Finally, the command can be persisted by writing it to ~/.xprofile file
echo "xrandr --output HDMI2 --left-of eDP1" > ~/.xprofile


For more information. look at this documentation:
https://wiki.ubuntu.com/X/Config/Resolution#Setting_xrandr_commands_in_.xprofile

Using Scanner in Ubuntu

After installing Epson scanner drivers from official Epson site, there are a few tools that can be used to scan:
1. scanimage terminal application
2. xsane GUI app
3. Simple Scan GUI app

The GUI apps are straight-forward to use. But I found that sometimes they just don't work. So I use the Terminal method when that happened :)

scanimage --resolution 300 --mode Color --format ppm > output.ppm

Can't create public-facing SSH server because of firewall or ISP policy? No worries, Reverse SSH Tunnel come to the rescue!

In regular situation, creating a public-facing SSH server, or any other server can be done as following:

Usually, the added complexity would simply be that ISP subscription only gives dynamic IP address, which gets randomized whenever the modem/connection is restarted. But this can easily be worked around using Dynamic DNS service.

However, today I encountered annoying issue. The ISP I used blocks their DMZ and Port Forwarding capabilities from their router! I called and they told me they had no solution! They also wouldn't let me tinker around with their router (i.e. installing Open-WRT or other open-source OS). But this turned out to be a great experience, because I learned a new cool SSH trick today!

 Reverse SSH Tunneling allows a relay-server that is publicly accessible to relay connection to private server that we have. So instead of SSH-ing to the private server, we SSH into the relay server, which in turn relay the connection to the private server. Wait, hold on a second! How could the relay-server establish a connection with the private server? Isn't it not publicly accessible? Simple, the connection between relay and private servers are initiated by the private server. So it's just like any computer can SSH into any publicly accessible SSH server out there.

The commands are the following:

# Run on private-server
# Tell nusantara-cloud.com SSH-server to relay any connection made to port 3559 of itself, to the private server.
ssh -R :3559:localhost:22 antonius@nusantara-cloud

# Editted on the relay-server:
# Without this configured, SSH-server would only relay connection made to port 3559 if it comes from loopback adapter. So this made it accessible from external world as well!
sudo echo "GatewayPorts clientspecified" >> /etc/ssh/sshd_config 

Credit: http://xmodulo.com/access-linux-server-behind-nat-reverse-ssh-tunnel.html

Dual GPU Laptop in Ubuntu. (NVIDIA and Intel HD5200)

I just bought myself a Thinkpad Yoga 14. I'm loving its IPS FHD screen which is accompanied with NVIDIA 840m GPU in addition to its onboard Intel HD5200, which more than enough to satisfy my occasional craving to game and do 3D rendering :)

Here are some useful stuffs I learned about dual GPU setup in Ubuntu.

1. NVIDIA isn't installed by default.
• It's running on Intel HD5200 by default because that's what's installed with the default Ubuntu setup.
• Instruction to install NVIDIA driver can be found here:
• Find out the latest driver that is compatible with your GPU.
• http://www.nvidia.com/Download/index.aspx
• To ease  out the process, install the driver via apt-get (e.g. sudo apt-get install nvidia-375 if NVIDIA page says that your latest driver is 375.66)
2. After your driver is installed according to step (1), you can switch between NVIDIA or onboard GPU via 'nvidia' program (open app search pane, type 'nvidia')

• Unfortunately, the new GPU to be used is only applied after a logout..
3. How to switch GPU without logging out or restarting?
1. Fire up terminal
2. sudo prime-switch nvidia  to switch to nvidia
3. sudo prime-switch intel  to switch to nvidia

Enjoy! I'll update this page as I find new tips and tricsk.

Source:

https://linuxconfig.org/how-to-install-the-latest-nvidia-drivers-on-ubuntu-16-04-xenial-xerus