Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts

Sunday, November 20, 2022

Download files from google drive with python - no API key

There are several scripts to download files from google drive using a shareable link. Some require
authentication others don't work. Here we're interested in those that do not require authentication.

The gdrivedl.py script provided by ndrplz just works fine.

> gdrivedl.py file_id

Another notable, currently non-working, alternative is googledrivedownloader. It probably worked at some point in time but today it fails. There's however a simple fix proposed in this thread. The code is here: 

import requests
 
def download_file_from_google_drive(id, destination):
    URL = "https://docs.google.com/uc?export=download"
 
    session = requests.Session()
 
    response = session.get(URL, params={"id": id, "confirm": 1}, stream=True)
    token = get_confirm_token(response)
 
    if token:
        params = {"id": id, "confirm": token}
        response = session.get(URL, params=params, stream=True)
 
    save_response_content(response, destination)
 
def get_confirm_token(response):
    for key, value in response.cookies.items():
        if key.startswith("download_warning"):
            return value
    return None
 
def save_response_content(response, destination):
    CHUNK_SIZE = 32768
 
    with open(destination, "wb") as f:
        for chunk in response.iter_content(CHUNK_SIZE):
            if chunk:  # filter out keep-alive new chunks
                f.write(chunk)
 
if __name__ == "__main__":
    file_id = "TAKE ID FROM SHAREABLE LINK"
    destination = "DESTINATION FILE ON YOUR DISK"
    download_file_from_google_drive(file_id, destination)

Thursday, September 2, 2021

Conditional formatting LaTeX tables depending on cell values

A package for generating conditional formatted LaTeX tables that extends the one proposed in a post by Siavoosh Payandeh Azad. I've just extended it with the command \gradientd for a divergent colormap. There are also several other solutions discussed in this stackexchange post. The example can be seen in Overleaf here.


The code for generating the table

\documentclass[table]{article}
\usepackage{highlight}

\begin{document}

% For a 2 color palette
% \gradientcell{cell_val}{min_val}{max_val}{colorlow}{colorhigh}{opacity} 

% For a 3 color/divergent palette
% \gradientcelld{cell_val}{min_val}{mid_val}{max_val}{colorlow}{colormid}{colorhigh}{opacity}

% Configure a shorthand macro
\newcommand{\g}[1]{\gradientcelld{#1}{-3}{0}{3}{red}{white}{green}{70}}

\begin{tabular}{l|ccccc}
{$\sigma$} &  40  &   80  & 100  &   200 &  800 \\
\hline
0  & \g{ 0.87} & \g{ 0.84} & \g{ 1.04} & \g{ 1.34} & \g{-4.30} \\
5  & \g{ 0.38} & \g{ 0.22} & \g{ 0.33} & \g{ 0.58} & \g{-2.18} \\
10 & \g{ 0.29} & \g{ 0.04} & \g{ 0.11} & \g{ 0.48} & \g{-0.67} \\
25 & \g{-0.05} & \g{-0.34} & \g{-0.39} & \g{-0.48} & \g{-0.53} \\
50 & \g{ 0.04} & \g{-0.18} & \g{-0.13} & \g{-0.08} & \g{ 0.11} \\
\end{tabular}

\end{document}

The package highlight.sty  implement these commands  

%% Siavoosh Payandeh Azad Jan. 2019
%% modified by Gabriele Facciolo Sep. 2021
\ProvidesPackage{highlight}[Cell background highlighting based on user data]
\RequirePackage{etoolbox}
\RequirePackage{pgf} % for calculating the values for gradient
\RequirePackage{xcolor} % enables the use of cellcolor make sure you have [table] option in the document class 

%======================================
% For a 2 color palette
% \gradientcell{cell_val}{min_val}{max_val}{colorlow}{colorhigh}{opacity} 
\newcommand{\gradientcell}[6]{
    % The values are calculated linearly between \midval and \maxval
    \ifdimcomp{#1pt}{>}{#3 pt}{\cellcolor{#5!100.0!#4!#6}#1}{
    \ifdimcomp{#1pt}{<}{#2 pt}{\cellcolor{#5!0.0!#4!#6}#1}{
         \pgfmathparse{int(round(100*(#1/(#3-#2))-(#2 *(100/(#3-#2)))))}
        \xdef\tempa{\pgfmathresult}
        \cellcolor{#5!\tempa!#4!#6}#1
    }}
 }
%======================================

%======================================
% For a 3 color/divergent palette % \gradientcelld{cell_val}{min_val}{mid_val}{max_val}{colorlow}{colormid}{colorhigh}{opacity} \newcommand{\gradientcelld}[8]{ \xdef\lowvalx{#2}% \xdef\midvalx{#3}% \xdef\maxvalx{#4}% \xdef\lowcolx{#5}% \xdef\midcolx{#6}% \xdef\highcolx{#7}% \xdef\opacityx{#8}% % The values are calculated linearly between \midval and \maxval \ifdimcomp{#1pt}{>}{\maxvalx pt}{\cellcolor{\highcolx!100.0!\midcolx!\opacityx}#1}{ \ifdimcomp{#1pt}{<}{\midvalx pt}{% \ifdimcomp{#1pt}{<}{\lowvalx pt}{\cellcolor{\midcolx!0.0!\lowcolx!\opacityx}#1}{ \pgfmathparse{int(round(100*(#1/(\midvalx-\lowvalx))-(\lowvalx*(100/(\midvalx-\lowvalx)))))}% \xdef\tempa{\pgfmathresult}% \cellcolor{\midcolx!\tempa!\lowcolx!\opacityx}#1% }}{ \pgfmathparse{int(round(100*(#1/(\maxvalx-\midvalx))-(\midvalx*(100/(\maxvalx-\midvalx)))))} \xdef\tempb{\pgfmathresult}% \cellcolor{\highcolx!\tempb!\midcolx!\opacityx}#1% }} }
%======================================

Friday, September 14, 2012

Writing floating point multi-channel TIFFs in Matlab

In a previous post I've commented on how to read a multi-channel floating point TIFF in Matlab using the Tiff class (available since R2009b).
The Tiff class also permits to write all TIFF flavors. But since this TIFF format is so flexible, setting up everything for writing a file is not straightforward. Concretely I'm interested in writing a multi-channel floating point TIFF. Using the details found here, I've wrote this functions that wraps all the parameter setup for writing this type of TIFF files.
function writeTIFF(data, filename)
% writeTIFF(data, filename)
% writes data as a multi-channel TIFF with single prec. float pixels
   t = Tiff(filename, 'w');
   tagstruct.ImageLength = size(data, 1);
   tagstruct.ImageWidth = size(data, 2);
   tagstruct.Compression = Tiff.Compression.None;
   %tagstruct.Compression = Tiff.Compression.LZW;        % compressed
   tagstruct.SampleFormat = Tiff.SampleFormat.IEEEFP;
   tagstruct.Photometric = Tiff.Photometric.MinIsBlack;
   tagstruct.BitsPerSample =  32;                        % float data
   tagstruct.SamplesPerPixel = size(data,3);
   tagstruct.PlanarConfiguration = Tiff.PlanarConfiguration.Chunky;
   t.setTag(tagstruct);
   t.write(single(data));
   t.close();
Sources: http://www.mathworks.com/matlabcentral/answers/7184#comment_15023http://www.mathworks.fr/help/techdoc/ref/tiffclass.hhttp://www.mathworks.fr/help/techdoc/ref/tiffclass.htmltml

Saturday, August 25, 2012

Change text inside multiple files (command line)

I always forget these commands, and they are so useful.
The sed command to replace some text in a set of files:
     sed -e 's/old text/new text/g'  -i .trash  *.c 

The regular expression 's/some text/new text/g' is applied to each line of each file *.c.
And the -i flag saves a backup of each file.

The regular expression language is powerful, for instance allows to remember parts of the pattern in the substitution. An example: the following command puts quotation marks around everything that follows the first = sign in each line of script.sh.
  sed -e 's/=\(.*\)/="\1"/' script.sh

The pattern enclosed between \(   and   \) is stored in \1, and then used in the replaced text.

Source: http://www.labnol.org/internet/design/wordpress-unix-replace-text-multiple-files/1128/

Sunday, May 13, 2012

Read floating point multi-channel TIFFs in Matlab >=R2009b

For reading a TIFF image in Matlab usually the imread function suffices:

d1 = imread('myfile.tif'); 

This call will even load single channel f32 (32bit floating point) images. But for multi-channel f32 images will fail. This is not a surprise, since most of the applications don't even load the single channel f32 files.

Starting from version R2009b Matlab includes a new Tiff class that implements much more of the TIFF format, allowing to read and write many flavors of TIFF files. The call is slightly different from imread: 

t = Tiff('myfile.tif'); 
d2 = t.read();

Sources: http://compgroups.net/comp.soft-sys.matlab/reading-64-bit-tif-image/405858
http://www.mathworks.fr/help/techdoc/ref/tiffclass.hhttp://www.mathworks.fr/help/techdoc/ref/tiffclass.htmltml

Sunday, April 29, 2012

MATLAB: "Trace/breakpoint trap" after system call

The MATLAB's system call permits to run a program in a shell. But some programs have dynamic dependencies that conflict with MATLAB's environment. For instance:

[status, result] = system(['/usr/local/bin/convert']); 

will throw something like:


dyld: Library not loaded: /opt/local/lib/libtiff.3.dylib
  Referenced from: /opt/local/bin/convert
  Reason: Incompatible library version: convert requires version 13.0.0 or later, but libtiff.3.dylib provides version 11.0.0

/usr/local/bin/convert: Trace/breakpoint trap 

A solution consists in resetting the variable DYLD_LIBRARY_PATH (for OSX or LD_LIBRARY_PATH in Linux)  before running the program. This call should do the trick:

[status, result] = system(['export DYLD_LIBRARY_PATH=""; ' '/opt/local/bin/convert']); 

Source: http://www.alecjacobson.com/weblog/?p=1453

Saturday, November 5, 2011

Minted: Highlighted source code in LATEX

Minted is a LaTeX package that facilitates expressive syntax highlighting using the powerful Pygments library. The package provides options to customize the highlighted source code output.
The example below shows how C code is highlighted,  and the output is shown at right:

\documentclass{article}
\usepackage{minted}

\begin{document}

\begin{minted}{c}
int main()
{
  printf("hello, world");
  return 0;
}
\end{minted}

\end{document}

Pygments also provides a command line interface pygmentize, among the possible output formats there are HTML, RTF, LaTeX and ANSI sequences.
For command line highlighting a solid alternative to Pygments is GNU Source-highlight.



Source: http://mirrors.ircam.fr/pub/CTAN/macros/latex/contrib/minted/minted.pdf
A couple of on-line highlighters: http://www.hilite.me/http://quickhighlighter.com/

Friday, November 4, 2011

Hack to prevent Caching of Dynamic Images

Web caches are a necessary component of the web. So, when developing dynamic web contents it is important to be sure if an object is cached or not, otherwise the user may end up seeing some old cached content. The objective here is to block caching of images produced by a service which generates a new image in each call, so the image must be transmitted every time.

The HTML meta tags do not apply to the case of images (and their effectivity is even questionable for HTML pages). The HTTP cache directives are better suited for cache control since most of the web caches honor them. But they entail some customization of the web server in order to send the correct header for each file.

The simplest way to prevent caching of a dynamic image, is to change its url at each run. For that, one possibility is to completely change the filename. But a trick allows to keep the original filenames and still prevent caching, it consists in appending a random querysting to the filename like this:

<img src="output.png?r=1234">

the querystring is discarded by the webserver where output.png is. Still it is seen as a different object by the cache, causing the image to be reloaded every time the page is loaded. 


Source: http://www.itsjustpoison.com/blog/2008/05/12/trick-prevent-image-caching-of-dynamic-images
Source: http://www.i18nguy.com/markup/metatags.html
Reading: http://www.mnot.net/cache_docs/

Thursday, September 15, 2011

MegaWave2 V3.01 - "Advanced - Ultimate" for OS X 10.6.8

I'm addressing again the installation of Megawave 2 V3.01, this time in OS X 10.6.8 (Snow Leopard).
A default installation of Snow Leopard uses a 32bits kernel, but which can run both: 32 and 64bits applications. This implies that all the system libraries are complied for both architectures.

Megawave however, is not 64bit-friendly so it mut be compiled as a 32bits application. The patches I've made are centered on enforcing a 32bits compilation at all stages of the Installation.
To avoid complications related the dependence on libtiff and libjpeg (which must be also compiled in 32bits), I've opted to include them in the package and compile them as part of Megawave installation.
The compiled libraries are stored inside Megawave library path sys/lib/ix86, so they do no interfere with other system libraries.

The resulting package only requires the gcc compiler, which can be either 4.0 or 4.2.

The installation should go as follows:

Wednesday, July 6, 2011

git branch in the shell

Add this to  ~/.bash_profile:

#Show the current git branch in the path
# \[\033[0;32m\] : sets color to green
# \[\033[0m\]    : reset to default color
PS1='\u@\h:\W\[\033[0;32m\]$( git branch 2> /dev/null | cut -f2 -d\* -s | sed "s/^ /#/" )\[\033[0m\]\$ '

the current git branch will be highlighted in the shell

username@hostname:directory#master

Bash colors: http://ubuntuforums.org/archive/index.php/t-31247.html

Monday, May 23, 2011

Fishtank VR project with OpenCV and OpenGL

A toy Fishtank VR application I've made using OpenCV. It uses head tracking to adjusts the view according to the position of the viewer (see related posts).
The OpenGL projection matrix was modified to allow rendering objects in front of the projection plane (the window) so that objects can "pop out" the screen.

TODO:
  • Stabilize the head tracking, using templated object search 
  • Calibrate the paralax factor, the apparent motion is not very realistic
  • Replace bitmap reading routines
  • Refactor 

Download the code. 
Builds with Xcode, but requires the OpenCV Framework to be installed. Also compiles with cmake in linux.

Similar projects:
http://www.tuaw.com/2011/04/11/ipad-2-gets-glasses-free-3d-display-using-front-facing-camera-fo/

Tuesday, February 1, 2011

Hamiltonian path count (Quora coding challenge)


I found the following programming challenge at quora.


Datacenter Cooling (Read more)

We have some rooms in our datacenter, and we need to connect them all with a single cooling duct.
Here are the rules:

The datacenter is represented by a 2D grid.
Rooms we own are represented by a 0.

Read more
.....
...


This problem amounts to count the number of Hamiltonian paths in a graph.
I wrote a small program to compute them, it runs in less than 45 seconds.
The code is (hidden) below.


Show the code.




Show the C++ code.