Friday, January 16, 2015

VIM + texlive + vim-latex suite + zathura +pdf + arch + so on

With a very little effort, one can learn to use the VIM editor. Once you are used to its keybindings, I am sure that no amount of development in touch screen or mouse technology will appeal to you. And the side effect is that you will frantically try to look for programs using similar keybindings for different purposes. This post is an example of such desperation.

Here, I will share my knowledge to set up an environment to complie .tex files into pdfs, while keeping it as lightweight as possible, while having access to several totally awesome features such as:
  • key binding of vim both in editor and viewer
  • vim-latex suite to reduce typing efforts
  • forward and inverse search
  • partial compilation
 All of the information presented here are available for a patient google user.

 Components used:
  1. Terminal: xterm
  2. TeX compiling package: texlive
  3. Editor: vim
  4. Plugin: vim-latex suite
  5. Output Format: pdf
  6. PDF viewer: zathura
 1. Installation of packages:

pacman -S texlive-core texlive-bin texlive-science gvim zathura zathura-pdf-poppler

Note: replace vim by gvim

(a) texlive package:
        Minimal: texlive-core
        Probably needed: texlive-bin, texlive-science

(b) vim editor:
        Obviously: vim
        For this purpose, we need to install gvim to enable +serverclient feature. and replace vim by gvim, although we will not use gvim. I could not find any other way but to recompile vim from source which seemed like a tremendous waste of time when installing gvim took very less time. Agreed, it is not cntradictory to my "minimalistic" approach, but, oh, well.

(c) pdf viewer:
        obviously: zathura
        For viewing pdfs: zathura-pdf-poppler
This is an excellent viewer with vim-like key binding. supports scrolling also. Capable of viewing djvu and ps as well. (install zathura-djvu, zathura-ps). Also, it supposedly supports plugins which I am yet to check out.


Now that you have all the tools needed to create a tex file and compile it using
pdflatex .tex
and view it in pdf viewer using
zathura .pdf
to get a pdf file, give it a try to see if everything works.


 2. Installing vim plugin vim-latex suite:

link: http://sourceforge.net/projects/vim-latex/

(a) Download the compressed files and extract it.
(b) Copy the content to ~/.vim
cp  ~/.vim/ftplugin/latex-suite/texrc ~/.vim/ftplugin/tex.vim

(c) The config file for the vim-latex suite will be ~/.vim/ftplugin/tex.vim
This file does not exist, but we can create it. This will have precedence over the file ~/.vim/ftplugin/latex-suite/texrc, which is the actual config file. But for our   need, we will modify it.


No action required for this step. We can take care of it in due time.

(d) Add the following lines to ~/.vimrc:

"""""""""""""""""""""""""""""""""""""""""""""""""
" REQUIRED. This makes vim invoke Latex-Suite when you open a tex file.
filetype plugin on

" IMPORTANT: grep will sometimes skip displaying the file name if you
" search in a singe file. This will confuse Latex-Suite. Set your grep
" program to always generate a file-name.
set grepprg=grep\ -nH\ $*

" OPTIONAL: This enables automatic indentation as you type.
"filetype indent on

" OPTIONAL: Starting with Vim 7, the filetype of empty .tex files defaults to
" 'plaintex' instead of 'tex', which results in vim-latex not being loaded.
" The following changes the default filetype back to 'tex':
let g:tex_flavor='latex'
"""""""""""""""""""""""""""""""""""""""""""""""""


(e) Optionally, add (create first, possibly, and) these lines to the file ~/.vim/ftplugin/tex.vim:

"""""""""""""""""""""""""""""""""""""""""""""""""
" this is mostly a matter of taste. but LaTeX looks good with just a bit
" of indentation.
set sw=2
" TIP: if you write your \label's as \label{fig:something}, then if you
" type in \ref{fig: and press you will automatically cycle through
" all the figure labels. Very useful!
set iskeyword+=:
"""""""""""""""""""""""""""""""""""""""""""""""""



At his moment, you should be able to utilise all the typing functionalities of vim-latex  while editing a .tex file. For example, type equation and press to see if you get 

\begin{equation}
 <++>
\end{equation}

etc. 

3. Setting up the default compiler and viewer:

If you follow the previous steps of installations, I can safely assume the following:

  • SyncTex is installed. Check if /usr/bin/synctex exists. If it is not installed, have to figure out how to.
  • Vim has +clientserver module enabled (made it sure by installing gvim). To check that it works, open vim and type: :version. There should be a '+' sign before  clientserver.
Create the file ~/.vim/ftplugin/tex.vim 
If it does not exist and add the following  lines to it:

"""""""""""""""""""""""""""""""""""""""""""""""""
" Default compiling format
let g:Tex_DefaultTargetFormat='pdf'


" Never Forget, To set the default viewer:: Very Important
let g:Tex_ViewRule_pdf = 'zathura'

" Trying to add same for pdfs, hoping that package SynTex is installed
let g:Tex_CompileRule_pdf = 'pdflatex -synctex=1 -interaction=nonstopmode $*'


" Get the correct servername, which should be the filename of the tex file,
" without the extension.
" Using the filename, without the extension, not in uppercase though, but
" that's okay for a servername, it automatically get uppercased
let theuniqueserv = expand("%:r")

" Working!!!, when we run vim appropriately
let g:Tex_ViewRuleComplete_pdf = 'zathura -x "vim --servername '.theuniqueserv.' --remote +\%{line} \%{input}" $*.pdf 2>/dev/null &'

" I like my sums and limits to have placeholders
let g:Tex_Com_sum = "\\sum\\limits\_{<++>}\^{<++>}<++>"
let g:Tex_Com_cap = "\\bigcap\\limits\_{<++>}\^{<++>}<++>"
let g:Tex_Com_cup = "\\bigcup\\limits\_{<++>}\^{<++>}<++>"
let g:Tex_Com_lim = "\\lim\\limits\_{<++>}\^{<++>}<++>"

" Forward search
" syntax for zathura: zathura --synctex-forward 193:1:paper.tex paper.pdf
function! SyncTexForward()
        let execstr = 'silent! !zathura --synctex-forward '.line('.').':1:"'.expand('%').'" "'.expand("%:p:r").'".pdf'
        execute execstr
endfunction
nmap f :call SyncTexForward()

" To save and compile with one command \k (k=kompile) :)
" no need to launch the pdf along with this because zathura can refresh
" itself after every compilation produces a new pdf, so \k is enough
nmap k :w ll

" I will just reamp this \lv thing to \v just to be consistent with \k and \f
nmap v lv

"""""""""""""""""""""""""""""""""""""""""""""""""""""""


Reasons for each line is included in the comments.

 4. Launching vim with servername:

Now we create a custom command for this mini setup that we have created. We create  the file called vtex and paste the following lines:


#######################################################
# The file called vtex, make it better, make it recognize only tex files
# while pressing
for i in "$@"; do
    if [[ ${i} == *.tex ]]; then
        fileName=$i;
        # Strip off path and extension, and convert to uppercase.
        # although there is no need to turn into upper case
        serverName=$(sed -e 's:\(.*/\)\?\(.*\)\.tex:\U\2\E:' <<< $i)
        break;
    fi
done

if [[ ${serverName} == '' ]]; then
        echo 'not a tex file or no non-extension part, no server to create'
else
        # run in server mode
        # now we need to make sure the forward search connects to the right server
        # check in .vimrc or tex.vim to see if it is correct
        echo 'creating server called $serverName'
        exec vim --servername $serverName "$@"
fi

#######################################################

Make this file executable (chmod +x) and keep it in a place where your $PATH can find it to be executed. In my case, .myscr.

So, typing vtex .tex in terminal will launch vim with proper server name so that zathura can be linked with it for inverse search, as specified in the tex.vim file.


Hence, the setup is done.


 5. Taking advantage of this whole setup:

Syntax Folding:

\rf to collapse all the equns.
za to expand individual folds. How to expand all? Ans: zi


Compiling and viewing: \l

After creating a .tex file, pressing \ll should compile the tex to generate a pdf file along with some other files. If there is any error or warning, it will pop up in the vim window. You can hit to jump to line the corresponding to error and, correct it, and press \ll again to compile again. To jump between error window and main file, press .

After a successful compilation, pressing \lv will open zathura to show the pdf file.

Note: pressing \ll or \lv requires the user to be quick while typing these keys.

According to our setup, pressing \k will compile and open zathura at the same time.

Forward search: \f

Pressing \f will perform a forward search, i.e. highlight the region in the pdf file opened in zathura, corresponding to the current line of code. This will work only if there has been a successful compilation (i.e. a pdf file has been generated by pdflatex command).

Inverse Search: Ctrl+Click

After opening the pdf file in zathura through \l or \lv, clicking any line in the pdf file while holding down the Ctrl key will make the cursor in vim window jump to the line corresponding to it.

Partial Compilation:

You can select a chunk of lines in visual mode (Shift+v) and press \ll or \l to compile that part of the tex file along with the preambles. In this process, a temporary tex file is created which can be handled like just any other tex file. You can \ll, \l, \f etc and when you are done, simply save and exit to get back to the original tex file.



See the file 'ftplugin/latex-suite/wizardfuncs.vim' for a complete list of shortcuts.

Reducing typing efforts:

pressing F7 while the cursor is touching a word will turn it into a keyword along with the format. E.g.
frac becomes \frac{<++>}{<++>}<++>
word becomes \word{}<++>
etc.

Ctrl+j jumps through <++>


Useful: frac , textbf

I have added a few more in tex.vim for sum, cup, cap, limit etc.

You can select some text in visual mode (v) and then press and enter the desired keyword to turn it into the arguments of a keyword.

Useful: If you want to make some text bold, select the text in visual mode, press and type textbf and hit to make that text boldface.
Or, select a long expression through visual mode and press and type sum and to put a sum sign beofre that.

Shift+ will let you change the parent command that you are inside.

For inserting things like \begin{equation} ...\end{equation}, enter 'equation' and press . Shift+ works accordingly.

Section Shortcuts:
Typing SSE will turn into
\section{New Section}
which is a nice way to start a section. Others are:


SPA for part
SCH for chapter
SSE for section
SSS for subsection
SS2 for subsubsection
SPG for paragraph
SSP for subparagraph


Environment shortcuts:
Type EEQ to get an equation environment. Select some text and press ,eq to enclose it with an equation environment. (i. e. 'E' from insert mode transforms into 'Leader2' character ',' in visual mode)


Most useful:::
Greek Letter Mappings

Lower case
`a through `z expand to \alpha through \zeta.

Note: That's a "tick" and not an "apostrophie".

Upper case Greek:

`D = \Delta
`F = \Phi
`G = \Gamma
`Q = \Theta
`L = \Lambda
`X = \Xi
`Y = \Psi
`S = \Sigma
`U = \Upsilon
`W = \Omega

Expanding the arrangement situation:

`^   Expands To   \Hat{<++>}<++>
`_   expands to   \bar{<++>}<++>
`6   expands to   \partial
`8   expands to   \infty
`/   expands to   \frac{<++>}{<++>}<++>
`%   expands to   \frac{<++>}{<++>}<++>
`@   expands to   \circ
`0   expands to   ^\circ
`=   expands to   \equiv
`\   expands to   \setminus
`.   expands to   \cdot
`*   expands to   \times
`&   expands to   \wedge
`-   expands to   \bigcap
`+   expands to   \bigcup
`(   expands to   \subset
`)   expands to   \supset
`<   expands to   \le
`>   expands to   \ge
`,   expands to   \nonumber
`~   expands to   \tilde{<++>}<++>
`;   expands to   \dot{<++>}<++>
`:   expands to   \ddot{<++>}<++>
`2   expands to   \sqrt{<++>}<++>
`|   expands to   \Big|
`I   expands to   \int_{<++>}^{<++>}<++>

Must get used to it. As soon as possible.

 In addition the visual mode macros are provided:

`(  encloses selection in \left( and \right)
`[  encloses selection in \left[ and \right]
`{  encloses selection in \left\{ and \right\}
`$  encloses selection in $$ or \[ \] depending on characterwise or linewise selection

Use the Alt key to make inserting all the \left ... \right stuff very easy: see the help, does not work because Alt is being used somewhere in my system. MOving on.

Pressing the Alt+B however, converts the previous character into a \mathbf{} character which is super-useful. Figure out a way to turn it into \mathbb{} which I use more often. Shouldn't be hard. Or, make your own. Have to find out how. It is in the Macro Customisation Section: http://vim-latex.sourceforge.net/documentation/latex-suite/customizing-macros.html



There is a referencing help section in completion section. Read later if you are into the \ref{equn:name} kind of things. Not useful for me now.

Pressing ... (3 dots) results in \ldots outside math mode and \cdots in math mode.


 Templates:

Insert a template in an empty tex file by :TTemplate
You can create your own template with <++> and required packages inside /ftplugins/latex-suite/templates.
Here is an example template I prefer to use usually:

File: ~/vim/ftplugins/latex-suite/templates/usual.tex

%%%%%%%%%%%%%%Please do not copy this line%%%%%%%%%%%%%
<+      +>      !comp!  !exe!
%        File: !comp!expand("%")!comp!
%     Created: !comp!strftime("%a %b %d %I:00 %p %Y ").substitute(strftime('%Z'), '\<\(\w\)\(\w*\)\>\(\W\|$\)', '\1', 'g')!comp!
% Last Change: !comp!strftime("%a %b %d %I:00 %p %Y ").substitute(strftime('%Z'), '\<\(\w\)\(\w*\)\>\(\W\|$\)', '\1', 'g')!comp!
%

% Documentclass
\documentclass{article}

% To use \text{} inside math mode
\usepackage{amsmath}
% To use \newtheorem styles
\usepackage{amsthm}
% To use \mathbb{} for fancy fonts
\usepackage{amsfonts}

% User defined fun
\newtheorem{thm}{Theorem}
\newtheorem{defn}{Definition}
\newtheorem{result}{Result}
\newtheorem*{notation}{Notation}

\begin{document}

<++>

\end{document}
%%%%%%%%%%%%%%Please do not copy this line%%%%%%%%%%%%%%


Now, in order to use this template, after creating an empty tex file with vtex, type:
:TTemplate
then select the template to use, and done.

Conclusion: The whole article is too big to be read by anybody who wants to setup a quick latex-compiling environments. For this reason, I plan to add a quick-setup guide to the beginning of this post so that I can follow the steps blindly without rediscovering the whole wheel.
 

Tuesday, August 5, 2014

Very light http server: lighttpd

The first thing that I wanted to do after setting up an Access Point on my laptop's wifi (See the previous post) is to setup an http server which is extremely light. My intention was to display some text, maybe some nice picture or a file-list of a directory. So, here is the solution, with a punny name:

https://wiki.archlinux.org/index.php/lighttpd
pacman -S lighttpd



The config file is located at:  
/etc/lighttpd/lighttpd.conf
Check if config file's syntax is correct:  
lighttpd -t -f /etc/lighttpd/lighttpd.conf
Default location of index.html file, according to the default config:
/srv/http

To create the index.html file:
echo 'TestMe!' >> /srv/http/index.html
chmod 755 /srv/http/index.html


Starting and reloading:
systemctl start lighttpd
systemctl reload lighttpd


Enabling over boot:
systemctl enable lighttpd

For PHP, SSL etc, check out the documentation home page.

Creating an access point in your system

If you wish to share your LAN bandwidth through wifi or just share some file though your wifi in an ad-hoc manner (a reference to wInDoWs users), or I don't know, just to check things out, you might want to look at this project by OBLiQUE that makes it very simple: https://bbs.archlinux.org/viewtopic.php?id=162895

Download the create_ap directory from git or otherwise.
No NAT or bridge needed. The script has options to set them or not. There are important dependencies. Check README or the create_ap file to find out.

Running: 

systemctl disable netctl-auto@wlp6s0.service
# So that is does not conflict with "netctl-auto" managed wlan card wlp6s0

systemctl stop netctl-auto@wlp6s0.service

# So, disabling it stops it from happening over roboots
# Stopping it actually stops the service

ifconfig wlp6s0 up
# To bring the interface up (important)

Then traverse the directory and see help to set up aps.
For example, the share your LAN's (eth0) internet through wifi (wlp6s0) access point called "koboglo" with passphrase "lenovo1234", use:
./create_ap wlp6s0 eth0 koboglo lenovo1234 

I need to use --no-virt option to make it work depending on my wlan card. i.e.

./create_ap --no-virt wlp6s0 eth0 myaccesspt thebigpasswd

All about wifi

Most of the information presented here are taken from and are no richer than: https://wiki.archlinux.org/index.php/Wireless_network_configuration
But then again, it is a well known fact that setting up/reconfiguring the wifi on your laptop may be one of the most challenging tasks you had to go through at some point or the other in your life. At least I had to. For the first time when you get it working, you ask yourself, "How exactly did it work?" But it's the same as asking "How do magnets work?" and the answer always is "Magic".

So, when you are in the illusion that you have isolated the minimum number of steps to get it working again, you do not bother making them generic or don't care whether they work on other computers or wlan cards. You just stop asking questions, write the steps down and upload it to a place like a blog or something similarly eternal, hoping that you won't have to go through the same nightmare all over again. Until you buy a new laptop.

Anyways, the moral here is: life is too short to rediscover the steps to get your wifi working again.


Vague Theory:

Notation: We are using Arch Linux. The wlan card is: wlp6s0, the profile you want to connect to is: profile.

The program "netctl-auto" automates the connecting process.
"wifi-menu" creates the profile and keeps it in /etc/netctl.
"netctl-auto start/enable profile" attaches the profile to netctl and finally, 
"netctl-auto start/enable netctl-auto@wlp6s0.service" forces wlp6s0 to connect to the internet using the profile. 

This eases up several jobs: 


Setting up wifi for the first time:  
wifi-menu to create a profile,  
dhcpcd wlp6s0 to get ip through dhcp,  
systemctl enable dhcpcd.service to make it work over reboots.  

systemctl start netctl-auto@wlp6s0 to let netctl-auto handle the automatic connection,

systemctl enable netctl-auto@wlp6s0 to make it work over reboots. netctl-auto enable profile to make the profile work over reboots. 

Deleting a profile, while it is already enabled: 

The first step is to tell netctl-auto to stop using the profile. 

netctl-auto disable profile  
should do that. But unfortunately, it still uses the profile somehow. You can verify it by issuing "systemctl restart netctl-auto@wlp6s0" and see that it is again connecting to the same profile. The solution is to delete the profile. 

rm /etc/netctl/profile  
netctl-auto disable profile
systemctl restart netctl-auto@wlp6s0 
and "ifconfig" to verify that the wlp6s0 is free. 

Note that: there is no such method to stop the profile using a single netctl-auto command. (Or, is there?)

Setting up a new profile:

To set up the profile, issue  
wifi-menu

If you get error (maybe through journalctl -xn) that "the interface is already up", it means that the profile still exists somehow. Follow the previous steps of "Deleting a profile while it is already enabled". 

After wifi-menu finishes creating the profile, if "systemctl enable/start netctl-auto@wlp6s0" is already issued, you should be able to connect. Otherwise, issue
systemctl start netctl-auto@wlp6s0 
systemctl enable netctl-auto@wlp6s0 

Optionally, use 
netctl-auto enable profile
to make it an automatic profile. 

Switching to a new profile: 

Use wifi-menu to create a new profile and use  
netctl-auto switch-to new_profile.

Saturday, March 29, 2014

From sound recording to voice/clap activated 9gag scrolling

Record from the microphone in wav format:
arecord out.wav
Or, more conveniently,
rec out.wav

The following commands require the package "sox".

The package sox provides almost everything regarding audio handling. Here is how you can monitor your microphone input without recording it (with a dB monitor, that too, stereo):
rec -n stat

Without the dB monitor:
rec -q -n stat

Show it for 5 seconds then stop:
rec -q -n stat trim 0 5

Repeat after every 5 milliseconds:
while true; do rec -q -n stat trim 0 0.05; done

Repeat after every 5 milliseconds, but only show the Maximum amplitude in that 5ms time interval:
while true; do echo $(rec -q -n stat trim 0 0.05 2>$1 | awk '/^Maximum amplitude/ { print $3}') ; done

Keep recording from standard output and echo 1 if the volume level is above certain threshold   (0.15ms), and return 0 otherwise:
while true; do echo $(rec -n stat trim 0 0.5 2>&1 | awk '/^Maximum amplitude/ { print $3 < .15 ? 0 : 1 }'); done

Following similar command pattern, the following code mutes the master audio level if the input level is more than the threshold, otherwise unmutes:
while true; do amixer set Master $(rec -n stat trim 0 0.5 2>&1 | awk '/^Maximum amplitude/ { print $3 < .15 ? 0 : 1 }'); done

Returns "high" if the amplitude is higher than the threshold, or returns "low":
while true; do ampli=$(rec -q -n stat trim 0 0.25 2>&1 | awk '/^Maximum amplitude/ { print $3 < 0.15 ? 0 : 1 }'); if [ $ampli -eq 1 ]; then echo "high"; else echo "low"; fi; done
(Self-note: Very mysteriously, $3 > 0.15 ? 1 :0 does not work. Why?)

Simulating key stokes:
Package called "xdotool" can send keystroked to X11 like this:
xdotool key k

So, here is the code to simulate a keypress of "j" simulated ny a tap on the mic or a clap:
while true; do ampli=$(rec -q -n stat trim 0 0.25 2>&1 | awk '/^Maximum amplitude/ { print $3 < 0.15 ? 0 : 1 }'); if [ $ampli -eq 1 ]; then xdotool key j; fi; done

This can be used while endlessly scrolling 9gag/facebook page without using hands. Best tool for a forever alone guy.

Friday, March 14, 2014

Splitting a long mp3 and tricks learned in the process

If you have a video, which contains a long audio that you want to split into multiple mp3 files using their time marker (created by you or found anywhere in a form of list), then you might want to proceed in the following way.

First, you have extract the audio in mp3 format by:
ffmpeg -i input_video_file -f mp3 out.mp3

Now the you have out.mp3 that has to be chopped up into pieces. For this, you need a package called mp3splt. Install that. Now you can chop it up manually if using the time markers. See manpage for details. But we are more interested in a more organised way. Generally, there are CUE files corresponding to audio CDs that can be found on the Internet. If you do not have a cue file, but you have the list of time markers and filename list (or song names) corresponding to them, then you can generate your cue sheet from the following website:
http://cuegenerator.net/

There, you need to insert the Tracklist (song name). Optionally, you can enter in  "songname - perfomer" format if you have the performer names also. It should look like:

songname1
songname2 - singer1
songnaem3 - singer2
songname4
songname5 - singer3


The Timings field has to be filled up in the suggested way (found in the "Help" link next to it). I followed the format: (for example)

Marker 01       00:00:00:00
Marker 02       00:04:12:00
Marker 03       00:09:56:00
.
.
.
Marker 22       01:12:56:00

Here, the time format aa:bb:cc:dd is given by hour:minute:second:milisecond. There are TABs after Marker and marker number, not spaces.

Creating strings following this certain formatting took a lot of time. I copied a list found in the about section of an youtube video in the following format:

time -- song_name -- optional_performer

I copied that into vim and these vim tricks came handy:

Creating visual blocks: Ctrl + v
Aligning text w.r.t. the character --:     :Align --

You need to have the plugin Align installed to use this command. It can be found here: http://www.vim.org/scripts/script.php?script_id=294

The installation instruction is given there. Just download the gz file. Then do vim file.gz. Then you are in vim file browser mode. do :so % and quit.

Insert a string str in front of every line in a visual block:

1. Select the visual block
2. Press I. It will take you to the beginning of the first string of the first line of the visual block.
3. Enter the string str.
4. Press Esc

Similarly, making any change in every line of a visual block involves similar steps.

~ When you have the strings organised in the desired way, copy them to the website to generate the cue file.

Once you enter the information, you can download the .cue file. Let's call it list.cue.

Now, we want to create a directory named inside which all the files will be stored with the following filenames:
-.mp3

I'm not interested in performers since that might increase the length of the filename.

So, we issue the following command in terminal:

mp3splt -c list.cue -o @b/@N-@t out.mp3

Here, -o sets the output format.
@b = Title of the album (director named @b will be created automatically)
@N = Track Number
@t = Song title

More formatting options are in man mp3splt.

Wednesday, March 5, 2014

Password protecting text files in VIM

To encrypt a file while creating:
vim -x filename
(You will be asked to enter a password twice)

To open the file:
vim filename
(You will be asked to enter the password)

To edit the file and save it with the same password:
vim filename
When finished editing, enter :wq!

To change the password:
vim +X filemane

Tip: to verify that the file is encrypted, always check by cat filename