pandoc: quite good conversion from Latex to docx with references and preserving tikz figures

This works surprisingly well, with all the figures and even rendering the BibTex references: 

> pandoc -C -s main.tex --natbib -o main.docx 

this will produce a document with citations using the  (Author, year) format, to produce citations with a numeric format download the ieee.csl file and call:  

> pandoc -C -s main.tex --natbib --csl=ieee.csl  -o main.docx  

To preserve tikz figures a specialized tikz-to-png filter is needed.  It should be downloaded in the current directory and use the call: 

> pandoc -C -s main.tex --natbib --csl=ieee.csl  --from latex+raw_tex --lua-filter=tikz-to-png.lua -o main.docx  

Figure numbers and captions are not preserved (for obscure reasons), however  table numbering can be  preserved using the document type  -t odt+native_numbering  or  -t docx+native_numbering. 

> pandoc -C -s main.tex --natbib --csl=ieee.csl  --from latex+raw_tex --lua-filter=tikz-to-png.lua -t docx+native_numbering -o main.docx  

Cross-references to sections and figures tables can be recovered with the resolve-references filter but the final format won't be great

> pandoc -C -s main.tex --natbib --csl=ieee.csl  --from latex+raw_tex --lua-filter=tikz-to-png.lua --lua-filter=resolve-references.lua -t docx+native_numbering -o main.docx  

Sources: 
https://waterprogramming.wordpress.com/2018/08/26/converting-latex-to-ms-word-docx-almost-perfectly/
https://tex.stackexchange.com/questions/268196/how-to-convert-latex-to-word-using-pandoc-and-keep-citations-as-numeral

https://tex.stackexchange.com/questions/431719/how-to-use-pandoc-to-derive-output-from-latex-and-tikz-to-a-docx-file

https://pandoc.org/lua-filters.html#building-images-with-tikz

https://github.com/jgm/pandoc/issues/5474

https://stackoverflow.com/questions/70087575/how-it-is-possible-to-convert-latex-document-to-word-docx-file-with-all-cross

Useful processes to kill in OSX

From time to time, some of user interface or system process go crazy or freeze. It's always good to know how to kill and restart those processes them without restarting the system.

• Frozen Finder: Kill it. Restarts by itself
  > sudo killall Finder

• Frozen Spotlight: Kill it. Restarts by itself
  > sudo killall SystemUIServer

• mDNSResponder sometimes gets stuck (observed in 10.5 and 10.6) and as it does so the system-wide name resolving stops working. Meaning that internet names are not resolved anymore
  > sudo launchctl unload /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
  > sudo launchctl load /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
  or ometimes it also works to kill it and restarts by itself
  > sudo killall mDNSResponder

• If mds goes crazy indexing some new volume: Stop it:
  > sudo mdutil -i off /path_to_volume
  or stop it on all volumes with:
  > sudo mdutil -a -i off

• ... any other?

Sources:
http://superuser.com/questions/231517/how-can-i-quit-frozen-spotlight-without-rebooting-my-computer
http://linuxtoosx.blogspot.fr/2011/10/restart-mdnsresponder.html
http://www.thexlab.com/faqs/stopspotlightindex.html
http://support.apple.com/kb/ht3789#

OSX: Start SSH socks proxy and change network settings from the command line

Everybody loves to travel. I also enjoy to experience the internet from other places, but this I can do without leaving my house. All I need is a remote unix computer which I can access by SSH.

The script below sets up SSH dynamic port forwarding and changes the network proxy settings all by itself. When communication is cut, with Ctrl-D, it goes back to the usual proxy configuration (no proxy). Tested on OSX 10.6.8.

#!/bin/bash
#title           :socksProxySSH.sh
#description     :initiates SSH dynamic port forwarding and sets up 
#                 the network proxy settings for you
#author          :Gabriele Facciolo (http://linuxtoosx.blogspot.com/)
#date            :20130615
#version         :1.0
#OSX version     :Tested on OSX 10.6.8  
#usage           :socksProxySSH.sh [remoteHost [localPort (default:1080)]]

DEFAULTSSHHOST=""
DEFAULTLOCALSOCKSPORT=1080

# DEAL WITH COMMAND LINE PARAMETERS
if [ "${1}" != "" ]
then
   SSHHOST=$1
else
   SSHHOST=$DEFAULTSSHHOST
fi
if [ "${2}" != "" ]
then
   LOCALSOCKSPORT=$2
else
   LOCALSOCKSPORT=$DEFAULTLOCALSOCKSPORT
fi
# SORRY NO SSHHOST
if [ "${SSHHOST}" == "" ]
then 
   echo "${0}: initiates SSH dynamic port forwarding and "
   echo "sets up the network proxy settings for you."
   echo ""
   echo "    Usage:   ${0} [remoteHost [localPort(default:1080)]]"
   echo ""
   echo "Indicate a remoteHost, or set the DEFAULTSSHHOST variable" 
   exit 1
fi


# ENABLE SOCKS ON ACTIVE INTERFACES 
networksetup -listallnetworkservices | grep -v asterisk | while read line ; do 
t=`networksetup -getinfo "$line" | grep "IP address" | grep -v none`
if [ "$t" ]; then
   tt=`networksetup -getsocksfirewallproxy "$line" | grep -v Authenticated | grep "Enabled: No"` 
   if [ "$tt" ]; then
      echo Enabling SOCKS on $line
      networksetup -setsocksfirewallproxy "$line" localhost $LOCALSOCKSPORT
   fi
fi
done 


# OPEN CONNECTION AND INITIATE SOCKS SERVER
echo ""
echo "INITIATING SSH CONNECTION AND SOCKS SERVER..."
echo "========================================================="
echo "TO DISCONNECT FIRST HIT Ctrl-D, WAIT A SECOND THEN Ctrl-C"
echo "========================================================="
echo ""
ssh -C -D *:$LOCALSOCKSPORT $SSHHOST


# DISABLE SOCKS ON ACTIVE INTERFACES 
networksetup -listallnetworkservices | grep -v asterisk | while read line ; do 
t=`networksetup -getinfo "$line" | grep "IP address" | grep -v none`
if [ "$t" ]; then
   tt=`networksetup -getsocksfirewallproxy "$line" | grep -v Authenticated | grep "Enabled: Yes"` 
   if [ "$tt" ]; then
      echo Disabling SOCKS on $line
      networksetup -setsocksfirewallproxystate "$line" off
   fi
fi
done 

Description of the key commands:

• networksetup -listallnetworkservices
  Displays a list of all the network services on the server's hardware ports. An asterisk (*) denotes that a network service is disabled.
  Sample output:
           USB Ethernet
           AirPort
           Bluetooth DUN

• networksetup -getinfo "Airport"
  Displays the IP address, subnet mask, router, and hardware address for the device that you specify.

• networksetup -getsocksfirewallproxy Airport
  Displays SOCKS proxy (server, port, enabled value) info for the device.
  Sample output:
           Enabled: No
           Server: localhost
           Port: 1080
           Authenticated Proxy Enabled: 0

• networksetup -setsocksfirewallproxy Airport localhost 1080
  Set and enable the SOCKS proxy server  localhost:1080.

•  ssh -C -D *:1080 your.remote.host
  Launch ssh with dynamic port forwarding

• networksetup -setsocksfirewallproxystate Airport off
  Disables SOCKS proxy.

Sources:

http://stackoverflow.com/questions/6796442/change-network-proxy-settings-in-via-commandline-in-mac-os-x-lion

http://linuxtoosx.blogspot.fr/2011/11/ssh-dynamic-port-forwarding-socks.html

https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/networksetup.8.html

Single TAB bash completion

Bash completion (when enabled) allows you to complete paths, commands, and filenames by hitting the TAB key. On the first press of the TAB key it completes the command as long as there is no ambiguity about the completion. If there are several possible completions a second key press reveals all the options.

It is possible to setup bash/readline to reveal the options after the first key press. Just add the following line to your ~/.inputrc :
 set show-all-if-ambiguous on

Source: http://blog.sanctum.geek.nz/lazier-tab-completion/

Change text inside multiple files (command line)

I always forget these commands, and they are so useful.
The sed command to replace some text in a set of files:
     sed -e 's/old text/new text/g'  -i .trash  *.c 

The regular expression 's/some text/new text/g' is applied to each line of each file *.c.
And the -i flag saves a backup of each file.

The regular expression language is powerful, for instance allows to remember parts of the pattern in the substitution. An example: the following command puts quotation marks around everything that follows the first = sign in each line of script.sh.
  sed -e 's/=\(.*\)/="\1"/' script.sh

The pattern enclosed between \(   and   \) is stored in \1, and then used in the replaced text.

Source: http://www.labnol.org/internet/design/wordpress-unix-replace-text-multiple-files/1128/

Homebrew package manager for OSX + Tips

Forget Fink and MacPorts, Homebrew is the miracle package manager you are looking for.

Homebrew installs packages to their own directory and then symlinks their files into /usr/local.
Then there is no need to tweak the system's and compiler's search paths to reach the packages. Also the installation is extremely simple, and the syntax familiar:
  brew [search, install, remove] packagename

The big critic I read about was the lack of packages. Never noticed, probably because the number of packages have exploded during the last year. The packages are built locally, so compiling large applications may take considerable time.

Tips:

• Compile for multiple architectures. If, for some reason, you need to compile a program/library with multiple architecture support, like on a 64bits system but with 32bits legacy applications. Then, while installing use the --universal flag:
       brew install --universal -v libtiff
  it will compile a "fat" library with multiple architectures (env UNIVERSAL_ARCH="i386 x86_64" to specify which architectures).

• Python. This setting is very important, but may depend on the version of OSX.
  When I installed Homebrew it did not compile its own python, instead it used the python that came with the system. Then all python packages installed with Homebrew are stored in: /usr/local/lib/python2.6/site-packages,  which is not in the search path for  python. It must be specified with the environment variable PYTHONPATH, by adding this line to ~/.bash_profile :
    # CONTRIBUTED PYTHON LIBRARIES 
    #------------------------------------------ 
    export PYTHONPATH=/usr/local/lib/python2.6/site-packages:$PYTHONPATH 
    #------------------------------------------
  
  More on Python and Homebrew.
• Big projects like GIMP, Inkscape, Firefox,  and many others...  Are not available as packages, they are already distributed as standalone OSX applications.

Find broken symbolic links

To list the symbolic links along with where they point to:

     find -L /PATH/TO/SEARCH -type l -exec ls -l {} \;  

Omitting  -exec ls -l {} \;  will just list the link name.

Other references: http://how-to.wikia.com/wiki/How_to_find_broken_symbolic_links

git branch in the shell

Add this to  ~/.bash_profile:

#Show the current git branch in the path
# \[\033[0;32m\] : sets color to green
# \[\033[0m\]    : reset to default color
PS1='\u@\h:\W\[\033[0;32m\]$( git branch 2> /dev/null | cut -f2 -d\* -s | sed "s/^ /#/" )\[\033[0m\]\$ '

the current git branch will be highlighted in the shell

username@hostname:directory#master$ 

Sources: nil and http://asemanfar.com/Current-Git-Branch-in-Bash-Prompt

Bash colors: http://ubuntuforums.org/archive/index.php/t-31247.html

Configure SSH jump hosts

Here is a configuration for establishing an ssh connection to a hidden host via a jump host.
This allows to connect hiddenhost just by typing (also works with scp and sftp)
    > ssh hiddenhost
if public keys for both jump.host and 10.0.0.1 are available in the local host,
then the connection will be established without prompting for any password.
Edit: .ssh/config and add the entry:


# connect: username@hiddenhost (10.0.0.1) via me@jump.host
Host hiddenhost
   User username
   HostName 10.0.0.1
   ProxyCommand ssh  me@jump.host  nc %h %p
   ForwardAgent yes

Alternatively with the following line is possible to establish the connection to 10.0.0.1, 

but the local key will not be presented to 10.0.0.1

    > ssh -t me@jump.host   ssh  username@10.0.0.1

Source: http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts#Jump_Hosts_--_Passing_through_a_gateway_or_two

Create user account from the OS X command line

Here is a procedure for adding a new user in OS X from the command line.
The problem is that there is no useradd or adduser in OS X, so the user must
be added to the Directory Service by hand.

1. First make sure to find an unused uid (unique ID) for the user,
   dscl . -list /Users UniqueID
   will list all the existing users, an unused number above 500 is good.
   
   In general to list a property from the directory use:
   dscl . -list {users|groups} {u|g}id
   or to see the id's of an existing user use:
   id auser
   
   
2. The Add the directory entry:
   dscl . -create /Users/luser
   dscl . -create /Users/luser UserShell /bin/bash
   dscl . -create /Users/luser RealName "Lucius Q. User"
   dscl . -create /Users/luser UniqueID "503"
   dscl . -create /Users/luser PrimaryGroupID 20
   dscl . -create /Users/luser NFSHomeDirectory /Users/luser
   
   The PrimaryGroupID=20 is usually staff. 80 is admin, it can be added with:
   dscl . -append /Groups/admin GroupMembership luser
   
   
3. Set the password:
   dscl . -passwd /Users/luser password
   or
   passwd luser
   
   
4. Finally, create the home directory with the appropriate permissions:
   mkdir /Users/luser
   chown luser:staff luser
   or even better, create and populate the directory with
   createhomedir -c -u luser
   
   
   
   

Further reading: http://serverfault.com/questions/20702/how-do-i-create-user-accounts-from-the-terminal-in-mac-os-x-10-5

Connecting to WiFi Network with hidden SSID

In order to connect to a network with hidden SSID (i.e. that does not broadcast it) in OS X 10.5 (in 10.6.8 this issue has been solved) the network must be configured each time we want to connect, even if the network appears in the advanced configuration preferences.

To the moment the best solution I've found is to forces the connection with the following shell command:
sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport --associate=yourHiddenSSID --password=yourkey

Source: http://superuser.com/questions/43836/automatically-connecting-to-hidden-ssid-wifi-network

vm_stat: virtual memory statistics (free memory)

The vm_stat utility displays the virtual memory usage


Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free:                   183268.
Pages active:                 363375.
Pages inactive:               262727.
Pages wired down:             176360.
.....

From this cryptic output the free memory is given by the free pages:
    183268 x 4096 bytes = 715 Mb

Source: http://arstechnica.com/apple/reviews/2001/10/macosx-10-1.ars/8

Code folding with Vim

I've learned about code folding using Vim.

The command zfa} collapses (folds) a C block using the } symbol as guide.

The following code maps the Spacebar to a function that folds/unfolds the current block (add this to ~/.vimrc), based on this.

"Fold coloring 

hi Folded        ctermfg=white ctermbg=NONE

fun! ToggleFold()

if foldclosed('.') < 0

". foldclose

:execute "normal zfa}<cr>"

else

. foldopen

endif

" Clear status line

echo 

endfun

" Map fold/unfold function to Space key.

"nmap <space> zfa}<cr>

"nmap <space> :call ToggleFold()<cr>

noremap <space> :call ToggleFold()<cr>

Here there is a more sophisticated function for folding.

vmap <space> zf

" inspired by Max Ischenko's http://www.vim.org/tips/tip.php?tip_id=108

function ToggleFold()

   if foldlevel('.') == 0

      " No fold exists at the current line,

      " so create a fold based on braces

      let x = line('.')    " the current line number

      normal 0

      call search("{")

      normal %

      let y = line('.')    " the current line number

      " Create the fold from x to y

      execute x . "," . y . " fold"

   else

      " Delete the fold on the current line

      normal zd

   endif

endfunction

nmap <space> :call ToggleFold()<cr>

Symbolic links and aliases

Aliases* and symbolic links do essentially the same thing but behave differently, one of the main problems with the formers (alias) is that they cannot be navigated from the terminal. On the other hand symbolic links can be navigated both from the terminal and Finder. Fortunately there is a way to create symbolic links in the Finder via AppleScript.

(*) The aliases behave like hard links, the source file can be moved but the alias always points to the same filesystem object. But they are more powerful than a hard link, with an alias you can link a folder or an object in another filesystem.

Bash completion

Bash completion is not enabled by default in OS X.

Use fink to install the completion files:

apt-get install bash-completion

or with homebrew:

brew install bash-completion

Source: http://mult.ifario.us/p/getting-bash-completion-magic-on-os-x

OpenTerminalHere.app

Opens a terminal on the folder currently shown in Finder.

Download it from:

http://henrik.nyh.se/2007/10/open-terminal-here-and-glob-select-in-leopard-finder

http://maururu.net/2007/enhanced-open-terminal-here-for-leopard/

Kgggghhhhaaaa!!! seq is MISSING!

Seems that jot is the replacement (and is more powerful) but has a different syntax.

Derived from:

http://codesnippets.joyent.com/posts/show/1797

http://www.labrat.info/blog/?p=9

Add this function to your ~/.bash_profile

to rewrite your simple seq calls as jot.

# integer test

function is_int() { return $(test "$@" -eq "$@" >/dev/null 2>&1); }

function seq() {

declare incr n1 n2 num1 num2

if [[ $# -eq 1 ]]; then

if ! $(is_int "$1"); then echo 'No integer!'; return 1; fi

for ((i=1; i<=${1}; i++)) { printf "%d\n" ${i}; }

elif [[ $# -eq 2 ]]; then

if ! $(is_int "$1") || ! $(is_int "$2"); then echo 'Not all arguments are integers!'; return 1; fi

if [[ $1 -eq $2 ]]; then

echo $1

elif [[ $2 -gt $1 ]]; then

for ((i=${1}; i<=${2}; i++)) { printf "%d\n" ${i}; }

elif [[ $1 -gt $2 ]]; then

for ((i=${1}; i>=${2}; i--)) { printf "%d\n" ${i}; }

fi

elif [[ $# -eq 3 ]]; then

num1=${1}

incr=${2}

num2=${3}

#/usr/bin/awk -v n1=${num1} -v n2=${num2} -v add=${incr} 'BEGIN{ for(i=n1; i<=n2; i+=add) print i;}' | /usr/bin/sed 's/.+e.+/0/'

/usr/bin/awk -v n1=${num1} -v n2=${num2} -v add=${incr} 'BEGIN{ for(i=n1; i<=n2; i+=add) print i;}' | /usr/bin/sed -E '/e/s/^.+e.+$/0/'

fi

return 0

}

-------------------------------------------------------------------------------------

UPDATED VERSION

-------------------------------------------------------------------------------------

function seq() {

# rewrites seq as jot for osx with format support.

# Updated version by G. Facciolo

if [ $# -gt 0 ]

then

if [ $1 == "-f" ]

then # with format parameters start from 3

format=$2

case $# in

3) # single argument

jot -w $format $3

return 0;

;;

4) # double argument

let nr="2*($4-$3)"

jot -w $format $nr $3 $4 1

return 0;

;;

5) # triple argument

let nr="2*($5-$3)/$4";

jot -w $format $nr $3 $5 $4

return 0;

;;

esac

else # without format parameters start from 1

format=""

case $# in

1) # single argument

jot $1

return 0;

;;

2) # double argument

let nr="2*($2-$1)";

jot $nr $1 $2 1

return 0;

;;

3) # triple argument

let nr="2*($3-$1)/$2";

jot $nr $1 $3 $2

return 0;

;;

esac

fi

else

echo "seq to jot for osx by G. Facciolo"

echo "Usage: seq [-f format] LAST"

echo " or: seq [-f format] FIRST LAST"

echo " or: seq [-f format] FIRST INCREMENT LAST"

return 1;

fi

}

-------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------

How to emulate Unix’ getent with MacOSX’s dscl

From: http://zzamboni.org/brt/2008/01/21/how-to-emulate-unix-getent-with-macosxs-dscl/

In linux there is:

getent passwd [uid number]

With dscl, you could do something like this:

dscl . -search /Users UniqueID [uid number]

dscl . -read /Users/[username]

Color Your OS X Command Prompt

Edit

~/.bash_profile

and add the following two lines:

export CLICOLOR=1

export LSCOLORS=ExFxCxDxBxegedabagacad

The new ls command does not use the same old --color=auto argument. If you want to set up the color labeling again, put the following in your ~/.tcshrc file (if you are using tcsh as your default shell):

setenv TERM "xterm-color"

setenv CLICOLOR "true"

setenv LSCOLORS "exfxcxdxbxegedabagacad"

The TERM variable can be set in the Terminal Preferences also, but it won't turn on the color labeling until you add at least the CLICOLOR variable to your .tcshrc file.

Read the rest of the hint for more on color definitions.

The colors can be set with the LSCOLORS variable. The color designators are as follows:

a black

b red

c green

d brown

e blue

f magenta

g cyan

h light grey

A bold black, usually shows up as dark grey

B bold red

C bold green

D bold brown, usually shows up as yellow

E bold blue

F bold magenta

G bold cyan

H bold light grey; looks like bright white

x default foreground or background

Note that the above are standard ANSI colors. The actual display may differ depending on the color capabilities of the terminal in use. The order of the attributes in the LSCOLORS variable is as follows:

1. directory
2. symbolic link
3. socket
4. pipe
5. executable
6. block special
7. character special
8. executable with setuid bit set
9. executable with setgid bit set
10. directory writable to others, with sticky bit
11. directory writable to others, without sticky bit

They are set in pairs, foreground (f) then background (b), i.e. fbfbfbfbfbfbfbfbfbfbfb for all 11 settings. The default is exfxcxdxbxegedabagacad, i.e. blue foreground and default background for regular directories, black foreground and red background for setuid executables, etc.

OS X VIM Terminal PageUp, PageDown, Home, End and Forward/Backward Word WORKING

One really big issue with the mac laptops is that some keys are missing.
Home, End, PageUp and PageDown are obtained as combination of Fn+Arrows.
And the forward/backward-word are not mapped by default they are Ctrl-Right/Left in linux.

Sometimes these keys are not properly reflected inside vim.

To correct this issue I found several pages, one of them is
http://blog.sbw.be/2008/12/11/mac-osx-vim-terminal-page-up-page-down-home-end-working/

Just go to:
Terminal > File > Preferences
and change the following keyboard settings.

OSX VIM TERMINAL PAGE UP, PAGE DOWN, HOME, END WORKING
Home: \033[1~
End: \033[4~
PageUp: \033[6~
PageDown: \033[5~

TERMINAL backward/forward-word
"\033[5D": backward-word
"\033[5C": forward-word

UPDATE: TERMINAL backward/forward-word ALSO WORKING WITH VIM
Change both: .inputrc and the terminal preferences to match these codes.
"\033[1;5C": forward-word
"\033[1;5D": backward-word