Sooo, I've just bought a Raspberry PI for a project, along with a $5 webcam for a Computer Vision experiment. I immediately tried the quality of the camera using "cheese" Linux program on my Ubuntu laptop, and it worked out fine. Unfortunately, in Raspberry PI, the program simply wouldn't detect the camera.
After hours of debugging, apparently the problem was with permission. The webcam enumerates as /dev/video0, which has 0660. Since "cheese" runs as "other", it doesn't have the permission to read it. Manually modifying it through "chmod o+rw /dev/video0" does the job. However, I want a more permanent and elegant solution.
The solution is to use udev. Here're the steps:
1. Get the webcam's vendorId and productId through lsusb/dmesg.
2. Get its DRIVER and SUBSYSTEM (properties required for udev setting) through "udevadm info -a -n [device_path]" (device_path is sth like /dev/video0)
3. Create new rules as instructed here: https://wiki.archlinux.org/index.php/Udev#Video_devices
4. As a reference, in my case the rule file is created here: /etc/udev/rules.d/83-webcam.rules, with the content being: KERNEL=="video[0-9]*", SUBSYSTEM=="video4linux", SUBSYSTEMS=="usb", ATTRS{idVendor}=="05e3", ATTRS{idProduct}=="0510", SYMLINK+="video-cam1", MODE="0666"
What's bold is the part that specifies the permission.
Hope this helps!
Random useful tech stuffs I write to remind myself. Hopefully useful for you!
Monday, October 3, 2016
Tuesday, December 1, 2015
Syncing Android Open Source Project Code Through Unreliable Connection
Since AOSP code is really big, there are multiple problems that can arise when syncing using slow and unreliable Internet connection. For this, the regular old "repo sync -j4" wouldn't work well.
So what are the issues?
1. Repo tries to download each of the branches available for each of the projects
This behavior is handy, but is undesired in a situation where Internet connection is slow.
To get repo to only sync the defined branch, use -c flag. For example repo sync -j4 -c
2. When a project failed to be downloaded, the whole sync is aborted.
You have been waiting for your sync for hours and suddenly your connection went down, and so the sync failed. You were hoping that you can just re-run the sync command again and continue where the it left off. Unfortunately, you found that the sync restarts!
To work around this, append "-f" to the repo command. This forces the sync to continue even if one or more projects fails.
3. Syncing with -f gets aborted because the computer is restarted or there is power outage.
Unfortunately, in this situation, repo doesn't partially download each of the projects. Even with -f flag, the projects would only be downloaded to the directory once it finishes fetching all the projects. In other words, when the sync is terminated abruptly, you'll have to restart it from the beginning!
To work around this, sync each of the projects individually using repo sync -j4 -c -f [project_name], where project_name can be found from the manifest file of interest.
I personally created a simple script to dump each of the paths that would be synced, and then did an xarg to sync each of them individually:
a. My scala script to dump each of the paths. I'm sure this can be done way way more efficiently using sed, but I'm just not familiar enough with it nor have the time to learn.
#!/bin/sh
exec scala "$0" "$@"
!#
import java.io.File;
import java.util.Scanner;
import java.util.List;
import java.util.ArrayList;
def getPath(s: Scanner) : List[String] = {
val list : List[String] = new ArrayList[String]()
def getPath(path : List[String], s : Scanner) : List[String] = {
if (!s.hasNextLine) return path
val line = s.nextLine
if (line.contains("path=")) {
val tmp = line.substring(line.indexOf("path=") + "path=".size)
val tmp2 = tmp.substring(1, tmp.indexOf("\"", 1))
if (!tmp2.equals("")) path.add(tmp2)
return getPath(path, s)
} else {
return getPath(path, s)
}
}
return getPath(list, s)
}
var f = new File("/home/aharijanto/tmp/cm12path.txt")
var s = new Scanner(f)
val l = getPath(s)
val it = l.iterator
while (it.hasNext) {
println(it.next)
}
Save the script about as repopath.scala, and then run it like:
./repopath.scala > ~/tmp/path.txt
2. Go to the directory where repo is init-ed
cat ~/tmp/path.txt | xargs -I xxx sh -c 'echo xxx > ~/tmp/log.txt ; repo sync -j4 -c -f xxx'
This way, in case of the sync being aborted, you just need to check on the log file and continue from the project that failed.
So what are the issues?
1. Repo tries to download each of the branches available for each of the projects
This behavior is handy, but is undesired in a situation where Internet connection is slow.
To get repo to only sync the defined branch, use -c flag. For example repo sync -j4 -c
2. When a project failed to be downloaded, the whole sync is aborted.
You have been waiting for your sync for hours and suddenly your connection went down, and so the sync failed. You were hoping that you can just re-run the sync command again and continue where the it left off. Unfortunately, you found that the sync restarts!
To work around this, append "-f" to the repo command. This forces the sync to continue even if one or more projects fails.
3. Syncing with -f gets aborted because the computer is restarted or there is power outage.
Unfortunately, in this situation, repo doesn't partially download each of the projects. Even with -f flag, the projects would only be downloaded to the directory once it finishes fetching all the projects. In other words, when the sync is terminated abruptly, you'll have to restart it from the beginning!
To work around this, sync each of the projects individually using repo sync -j4 -c -f [project_name], where project_name can be found from the manifest file of interest.
I personally created a simple script to dump each of the paths that would be synced, and then did an xarg to sync each of them individually:
a. My scala script to dump each of the paths. I'm sure this can be done way way more efficiently using sed, but I'm just not familiar enough with it nor have the time to learn.
#!/bin/sh
exec scala "$0" "$@"
!#
import java.io.File;
import java.util.Scanner;
import java.util.List;
import java.util.ArrayList;
def getPath(s: Scanner) : List[String] = {
val list : List[String] = new ArrayList[String]()
def getPath(path : List[String], s : Scanner) : List[String] = {
if (!s.hasNextLine) return path
val line = s.nextLine
if (line.contains("path=")) {
val tmp = line.substring(line.indexOf("path=") + "path=".size)
val tmp2 = tmp.substring(1, tmp.indexOf("\"", 1))
if (!tmp2.equals("")) path.add(tmp2)
return getPath(path, s)
} else {
return getPath(path, s)
}
}
return getPath(list, s)
}
var f = new File("/home/aharijanto/tmp/cm12path.txt")
var s = new Scanner(f)
val l = getPath(s)
val it = l.iterator
while (it.hasNext) {
println(it.next)
}
Save the script about as repopath.scala, and then run it like:
./repopath.scala > ~/tmp/path.txt
2. Go to the directory where repo is init-ed
cat ~/tmp/path.txt | xargs -I xxx sh -c 'echo xxx > ~/tmp/log.txt ; repo sync -j4 -c -f xxx'
This way, in case of the sync being aborted, you just need to check on the log file and continue from the project that failed.
Saturday, October 24, 2015
Understanding how Ubuntu Touch is forked out of AOSP
This post is meant for readers who have familiarity with AOSP building. As I just got my new Nexus 5 x, as a side project, I decided to try to port ubuntu-touch to my old phone: HTC One m7.
First of, Ubuntu touch is built on top of AOSP project. It's using a fork of Android kernel, frameworks, and build system. More precisely, the diagram below explains the architecture:
(note that I didn't draw the diagram, credit goes to Ubuntu's official page: https://developer.ubuntu.com/en/start/ubuntu-for-devices/porting-new-device/)
In the past, Ubuntu Touch project based the Android code they're using of Cyanogen's fork of AOSP (http://cyanogenmod.org), however they recently moved to pure AOSP version. To be precise, by the time of this writing, they're using version 4.4.2.r1. As for my device, since there's a Cyanogen support for it, I think ubuntu-touch can be ported with reasonable effort.
I think using AOSP instead of building the project from scratch is a really smart move. I can imagine that not only it will speed up their development process, but they may also benefit from improvements that Google are periodically making to the code. And the neat thing about AOSP code is that, the Java portion of it is pretty-well isolated from the the build system, making it a perfect fit for Ubuntu Touch that doesn't rely on Java.
Before we begin exploring the changes that Ubuntu Touch made atop of AOSP, I'd like to give a crash intro towards AOSP. First, it is built on top of a modified Linux kernel. Some substantial modifications that are worth to mention are wakelock-support and binder-IPC. The first provides userspace a way to prevent Linux to go to deep-sleep state or LP0. Unlike desktop, when a mobile device is put to sleep (e.g. power button is pressed), we may not want it go to deep-sleep right away. Sometimes we want the sleep to be delayed until a certain task is completed. (e.g. until a data synchronization is completed or until an email is sent). And for binder, it is a lightweight IPC mechanism that is used all over Android. It provides the needed security and flexibility that are expected by a mobile application. For example, when an IPC occurs, the caller identity can be retrieved, which in turns allows an application to see whether the caller has the appropriate permission required for the request. And in terms of flexibility, Google has build a very convenient userspace frameworks on top of its kernel driver implementation. This frameworks allows IPC communication between Java applications, native applications, and even Java-native applications. From my little observation so far, Ubuntu Touch seems to also rely on wakelock and binder.
Now, lets go back to the changes towards AOSP that Ubuntu Touch made. When Android kernel completes its initialization, it calls the very first process, init. This process then does all the userspace initialization, such as creating sysfs nodes and settings proper permissions, and kick-starting different daemons. What's necessary to know here is that init is the one that starts a Java virtual machine that Android heavily uses. init forks itself and runs zygote, which handles all Java-related stuffs. And as Ubuntu Touch does not use Java, a modification was made so zygote never runs. For more details, refer to the code under phablet/system/core. In particular, look for the following commit:
commit 1742098c17be31d968dce45a8eda4552602398c7
Author: Ricardo Salveti de Araujo <ricardo.salveti@canonical.com>
Date: Fri Jan 10 03:06:09 2014 -0200
init.rc: adding sensorservice and ubuntuappmanager
Also removing some unused services.
Change-Id: I7881bf436319ed09cbac46450c51710b6cf93c11
Signed-off-by: Ricardo Salveti de Araujo <ricardo.salveti@canonical.com>
[To be continued...]
1. upstart -> ./ubuntu/upstart-property-watcher/upstart-property-watcher.c:78:#define
http://upstart.ubuntu.com/
2.
1. upstart -> ./ubuntu/upstart-property-watcher/upstart-property-watcher.c:78:#define
http://upstart.ubuntu.com/
2.
Friday, October 23, 2015
CUDNN Benchmark Tools
I was trying to evaluate whether my embedded device, NVIDIA Shield TV, was capable of running a particular CNN learning algorithm in real-time. Instead of directly implementing the entire learning algorithm with the risk of not being able to meet the targeted performance, I decided to study the feasibility first by writing a benchmark tool. It runs forward convolutions with different configurable parameters (number of feature maps, batch size, filter size) and algorithm.
For those who are not familiar with CUDNN library, it supports different algorithms for running a forward convolution; some try to minimize GPU memory footprint, while some other focus on performance without worrying about memory-footprint. Perhaps the idea is to support different GPUs with different specifications (number of CUDA cores, memory size, etc), as well as different use-cases.
Sample output:
For those who are not familiar with CUDNN library, it supports different algorithms for running a forward convolution; some try to minimize GPU memory footprint, while some other focus on performance without worrying about memory-footprint. Perhaps the idea is to support different GPUs with different specifications (number of CUDA cores, memory size, etc), as well as different use-cases.
Sample output:
n, w, h, c, k, filter_dim, avg_time(us), max_time(us)
1, 3840, 2160, 1, 1, 3, 74383, 75142
1, 3840, 2160, 1, 1, 5, 88465, 88819
1, 3840, 2160, 1, 1, 9, 159752, 160324
Total time taken=322600 us.
Since the output is comma-separated, they can easily be parsed by spreadsheet processor.
Check the code out here:
https://github.com/blacksoil/CUDNN-BenchmarkUtilityThursday, October 22, 2015
Ubuntu 14.04 suddenly won't boot? This is how you debug it!
After a couple days without rebooting my X201 that was running Ubuntu, I decided to reboot the system. And unfortunately, it won't boot afterwards! After passing the Ubuntu's familiar purplish-black blank background, it got stuck in a black screen. What scared me were that pressing caps-lock didn't even turn on the keyboard indicator and I also couldn't enter "console mode" (ctrl + alt + f1). I was scared that the system went into a hard-hang, perhaps because I accidentally installed some craps. However, it turned out that my xserver binary was missing; the file /usr/bin/X didn't exist, for some reason! How did I figure that out? This post explains it.
Ubuntu keeps different the logs of the last boot attempt. These logs can be used to triage booting issue and potentially are useful to fixing it too. It Through these logs, I was able to figure out my xserver issue. This is how you do it:
1. Boot the system, get it to the hanging state. This step is necessary in order to get the log files generated.
2. Boot to a live-session using Ubuntu CD or Ubuntu USB stick
3. Open "[path to partition]/var/log" of the partition where the problematic Ubuntu is installed. Note that this is not just "/var/log" because that belongs to the currently running live session.
4. Sort the log files by modified date, and open the logs that correspond to the booting attempt from step 1.
5. As there are different log files, I'd suggest to first look at boot.log, which is similar to the kernel log that you get from dmesg. If nothing is suspicious there, then it's likely to be a userspace issue.
6. In my case, I found my xserver problem through the log at /var/log/lightdm/lightdm.log. This is the suspicious snippet of the logs:
+0.05s] DEBUG: DisplayServer x-0: Logging to /var/log/lightdm/x-0.log
[+0.05s] DEBUG: DisplayServer x-0: Can't launch X server X -core, not found in path
[+0.05s] DEBUG: DisplayServer x-0: X server stopped
Through a quick google search, I figured that /usr/bin/X is where "X server X -core" is supposed to be located at. In my case, the file was missing, so it was rather obvious. I then rebooted to Ubuntu recovery mode (by holding "shift" right after the system passes bios), and reinstalled xserver via sudo apt-get install --reinstall xserver-xorg.
Hope this post can be useful!
Monday, October 19, 2015
Qualcomm's thermal management file on Android devices.
I was reading through ubuntu-touch source code, in particular the /device/lge/mako repo, which is the configuration for Nexus 5 device, I came across thermald-mako.conf file. It looks like the following:
sampling 5000
[tsens_tz_sensor10]
sampling 5000
thresholds 60 120
thresholds_clr 57 115
actions none shutdown
action_info 0 5000
[batt_therm]
sampling 5000
thresholds 360 370 380 390 410 420 450
thresholds_clr 350 360 370 380 400 410 440
actions cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery
action_info 1512000+400000000+240+0 1296000+325000000+215+0 1296000+325000000+192+0 1188000+200000000+181+1 1188000+200000000+181+1 1188000+200000000+181+2 1188000+200000000+181+3
And from a quick google search, I found out (from an xda-developers thread) that the file is used to control thermal throttling based on the sysfs nodes here:
cat /sys/devices/virtual/thermald/thermald_zone*/temp
The number after tsens_tz_sensor is matched to the * on the path. The value read there is then used for thresholding.
How did I figure it's Qualcomm's? When I did a search over thermald on the repo, I found these:
aharijanto@aharijanto-ThinkPad-X201:~/droid/ubuntu/device/lge/mako$ g thermald
device.mk:60: device/lge/mako/thermald-mako.conf:system/etc/thermald.conf
init.mako.rc:214: # communicate with mpdecision and thermald
init.mako.rc:386:service thermald /system/bin/thermald
proprietary-blobs.txt:37:/system/bin/thermald
self-extractors/qcom/staging/device-partial.mk:37: vendor/qcom/mako/proprietary/thermald:system/bin/thermald:qcom \
self-extractors/extract-lists.txt:42: system/bin/thermald \
self-extractors/generate-packages.sh:104: system/bin/thermald \
vendor_owner_info.txt:152:system/bin/thermald:qcom
sampling 5000
[tsens_tz_sensor10]
sampling 5000
thresholds 60 120
thresholds_clr 57 115
actions none shutdown
action_info 0 5000
[batt_therm]
sampling 5000
thresholds 360 370 380 390 410 420 450
thresholds_clr 350 360 370 380 400 410 440
actions cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery cpu+gpu+battery
action_info 1512000+400000000+240+0 1296000+325000000+215+0 1296000+325000000+192+0 1188000+200000000+181+1 1188000+200000000+181+1 1188000+200000000+181+2 1188000+200000000+181+3
And from a quick google search, I found out (from an xda-developers thread) that the file is used to control thermal throttling based on the sysfs nodes here:
cat /sys/devices/virtual/thermald/thermald_zone*/temp
The number after tsens_tz_sensor is matched to the * on the path. The value read there is then used for thresholding.
How did I figure it's Qualcomm's? When I did a search over thermald on the repo, I found these:
aharijanto@aharijanto-ThinkPad-X201:~/droid/ubuntu/device/lge/mako$ g thermald
device.mk:60: device/lge/mako/thermald-mako.conf:system/etc/thermald.conf
init.mako.rc:214: # communicate with mpdecision and thermald
init.mako.rc:386:service thermald /system/bin/thermald
proprietary-blobs.txt:37:/system/bin/thermald
self-extractors/qcom/staging/device-partial.mk:37: vendor/qcom/mako/proprietary/thermald:system/bin/thermald:qcom \
self-extractors/extract-lists.txt:42: system/bin/thermald \
self-extractors/generate-packages.sh:104: system/bin/thermald \
vendor_owner_info.txt:152:system/bin/thermald:qcom
Saturday, October 10, 2015
Convolutional Neural Net (CNN) vs. Feed-foward Neural Net vs. Human Visual-Processing System.
Recently, I was working with on a digit-classification task. I trained a variant of LeNet CNN network on a modified MNIST dataset. The modification is the addition of non-digits samples, which are augmented as the following:
1. For each of the images in the dataset, create four clones, each shifted by 50% upward, downward, to the left, and to the right, respectively.
2. Each of the four images generated from (1) is translated again. An image that from step (1) was translated upward/downward is now shifted randomly (0-25%) to the right/left. And similarly, the one that was translated to the left/right from (1) is now shifted upward/downward.
The network itself is modified to output one more digit, which is used to classify a non-digit input. All the generated non-digits samples explained above are labeled with this digit.
After that, I was wondering if feed-forward neural net would produce better result when used in a sliding window fashion. My thinking is that since its connectivity is more straightforward, its activation units would require more "exact" matches to trigger, just as how it is less robust to translations when compared to CNN, and hence would be more robust to negative samples. When I get a chance, I'd really like to experiment with this. I'm thinking of deep visualization technique to analyse the robustness of CNN vs. feed-forward neural net against negative samples.
Also, it's worth to mention that I came across End-to-End Text Recognition with Convolutional Neural Networks [Wang, et al], which talks about text recognition using CNN. What's interesting is that they actually train a dedicated classifier, which is used in a sliding-window fashion, to detect whether a region contains a centered text or not. Only after the region is determined, they run another character-classifier that actually predicts the text there.
Another interesting finding is about how a deep neural network can be fooled. On their paper, Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, Nguyen, et al found that one can fool the state-of-art of pattern-recognition DNN with an augmented image (http://www.evolvingai.org/fooling). Their paper shows how an image can be augmented in a way that a DNN-based classifier would classify it to be of a particular pattern with a very high accuracy. The image can be assembled such that human would still be able to make out the classifier-claimed pattern or one that totally makes no sense (false-positive).
The need of explicitly training NN networks against negative samples got me thinking about how our (human) visual processing system works. It seems that we can make out a particular shape out of very complex background very easily. I'm interested to understand how our brain is actually achieving this. With this understanding, we'll be able to come up with a better architecture to tackle recognition problem, as well as to simplify the tedious process of gathering the training data. While gathering enough positive samples - in order to train a classifier that works reasonably well - is already a tedious work, gathering negative samples is even more challenging. If the size of the positive sample in the universe of all samples is X, the size of the negative counterpart is the size of the entire universe subtracted by X. Imagine how large it is!
1. For each of the images in the dataset, create four clones, each shifted by 50% upward, downward, to the left, and to the right, respectively.
2. Each of the four images generated from (1) is translated again. An image that from step (1) was translated upward/downward is now shifted randomly (0-25%) to the right/left. And similarly, the one that was translated to the left/right from (1) is now shifted upward/downward.
The network itself is modified to output one more digit, which is used to classify a non-digit input. All the generated non-digits samples explained above are labeled with this digit.
[Post the images that are falsely classified as digit here]
When used on an image of properly aligned digit, the trained classifier worked pretty well. When it was used in a sliding-window fashion, though, the result was disappointing. There are still so many false positives. It incorrectly classified what obviously seemed like non-digits, as digits. I suspected CNN architecture to be the culprit here. My thought is that since a convolution layer "combines" neighboring pixels together, multiple convolution layers followed by a fully-connected layer would "combine" more than just neighboring pixels, resulting in a potentially unexpected connectivity between pixels, and hence an unexpected activation.After that, I was wondering if feed-forward neural net would produce better result when used in a sliding window fashion. My thinking is that since its connectivity is more straightforward, its activation units would require more "exact" matches to trigger, just as how it is less robust to translations when compared to CNN, and hence would be more robust to negative samples. When I get a chance, I'd really like to experiment with this. I'm thinking of deep visualization technique to analyse the robustness of CNN vs. feed-forward neural net against negative samples.
Also, it's worth to mention that I came across End-to-End Text Recognition with Convolutional Neural Networks [Wang, et al], which talks about text recognition using CNN. What's interesting is that they actually train a dedicated classifier, which is used in a sliding-window fashion, to detect whether a region contains a centered text or not. Only after the region is determined, they run another character-classifier that actually predicts the text there.
Another interesting finding is about how a deep neural network can be fooled. On their paper, Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, Nguyen, et al found that one can fool the state-of-art of pattern-recognition DNN with an augmented image (http://www.evolvingai.org/fooling). Their paper shows how an image can be augmented in a way that a DNN-based classifier would classify it to be of a particular pattern with a very high accuracy. The image can be assembled such that human would still be able to make out the classifier-claimed pattern or one that totally makes no sense (false-positive).
The need of explicitly training NN networks against negative samples got me thinking about how our (human) visual processing system works. It seems that we can make out a particular shape out of very complex background very easily. I'm interested to understand how our brain is actually achieving this. With this understanding, we'll be able to come up with a better architecture to tackle recognition problem, as well as to simplify the tedious process of gathering the training data. While gathering enough positive samples - in order to train a classifier that works reasonably well - is already a tedious work, gathering negative samples is even more challenging. If the size of the positive sample in the universe of all samples is X, the size of the negative counterpart is the size of the entire universe subtracted by X. Imagine how large it is!
Subscribe to:
Posts (Atom)

