Showing posts with label network. Show all posts
Showing posts with label network. Show all posts

Sunday, November 20, 2022

Download files from google drive with python - no API key

There are several scripts to download files from google drive using a shareable link. Some require
authentication others don't work. Here we're interested in those that do not require authentication.

The gdrivedl.py script provided by ndrplz just works fine.

> gdrivedl.py file_id

Another notable, currently non-working, alternative is googledrivedownloader. It probably worked at some point in time but today it fails. There's however a simple fix proposed in this thread. The code is here: 

import requests
 
def download_file_from_google_drive(id, destination):
    URL = "https://docs.google.com/uc?export=download"
 
    session = requests.Session()
 
    response = session.get(URL, params={"id": id, "confirm": 1}, stream=True)
    token = get_confirm_token(response)
 
    if token:
        params = {"id": id, "confirm": token}
        response = session.get(URL, params=params, stream=True)
 
    save_response_content(response, destination)
 
def get_confirm_token(response):
    for key, value in response.cookies.items():
        if key.startswith("download_warning"):
            return value
    return None
 
def save_response_content(response, destination):
    CHUNK_SIZE = 32768
 
    with open(destination, "wb") as f:
        for chunk in response.iter_content(CHUNK_SIZE):
            if chunk:  # filter out keep-alive new chunks
                f.write(chunk)
 
if __name__ == "__main__":
    file_id = "TAKE ID FROM SHAREABLE LINK"
    destination = "DESTINATION FILE ON YOUR DISK"
    download_file_from_google_drive(file_id, destination)

Tuesday, May 25, 2021

Blockchain explained

The clearest explanation of blockchain I've seen, from Anders Brownworth and it comes with an online demo. 






Friday, February 12, 2021

nettop: monitor the per-process bandwidth usage in OSX

This command will display the network usage of all the running processes


   nettop -P -k state,interface -d -s 3


Flags explained:

-s refreshes every 3 seconds

-P collapses the rows of each parent process

-k state,interface removes less informative columns that stand between you and the bytes in/out columns

-d activates the delta option (same as pressing the d button)

Use the h button or run man nettop for some more options.


source: https://apple.stackexchange.com/questions/16690/how-can-i-see-what-bandwidth-each-app-or-process-is-using

Saturday, June 15, 2013

OSX: Start SSH socks proxy and change network settings from the command line

Everybody loves to travel. I also enjoy to experience the internet from other places, but this I can do without leaving my house. All I need is a remote unix computer which I can access by SSH.

The script below sets up SSH dynamic port forwarding and changes the network proxy settings all by itself. When communication is cut, with Ctrl-D, it goes back to the usual proxy configuration (no proxy). Tested on OSX 10.6.8.


#!/bin/bash
#title           :socksProxySSH.sh
#description     :initiates SSH dynamic port forwarding and sets up 
#                 the network proxy settings for you
#author          :Gabriele Facciolo (http://linuxtoosx.blogspot.com/)
#date            :20130615
#version         :1.0
#OSX version     :Tested on OSX 10.6.8  
#usage           :socksProxySSH.sh [remoteHost [localPort (default:1080)]]

DEFAULTSSHHOST=""
DEFAULTLOCALSOCKSPORT=1080

# DEAL WITH COMMAND LINE PARAMETERS
if [ "${1}" != "" ]
then
   SSHHOST=$1
else
   SSHHOST=$DEFAULTSSHHOST
fi
if [ "${2}" != "" ]
then
   LOCALSOCKSPORT=$2
else
   LOCALSOCKSPORT=$DEFAULTLOCALSOCKSPORT
fi
# SORRY NO SSHHOST
if [ "${SSHHOST}" == "" ]
then 
   echo "${0}: initiates SSH dynamic port forwarding and "
   echo "sets up the network proxy settings for you."
   echo ""
   echo "    Usage:   ${0} [remoteHost [localPort(default:1080)]]"
   echo ""
   echo "Indicate a remoteHost, or set the DEFAULTSSHHOST variable" 
   exit 1
fi


# ENABLE SOCKS ON ACTIVE INTERFACES 
networksetup -listallnetworkservices | grep -v asterisk | while read line ; do 
t=`networksetup -getinfo "$line" | grep "IP address" | grep -v none`
if [ "$t" ]; then
   tt=`networksetup -getsocksfirewallproxy "$line" | grep -v Authenticated | grep "Enabled: No"` 
   if [ "$tt" ]; then
      echo Enabling SOCKS on $line
      networksetup -setsocksfirewallproxy "$line" localhost $LOCALSOCKSPORT
   fi
fi
done 


# OPEN CONNECTION AND INITIATE SOCKS SERVER
echo ""
echo "INITIATING SSH CONNECTION AND SOCKS SERVER..."
echo "========================================================="
echo "TO DISCONNECT FIRST HIT Ctrl-D, WAIT A SECOND THEN Ctrl-C"
echo "========================================================="
echo ""
ssh -C -D *:$LOCALSOCKSPORT $SSHHOST


# DISABLE SOCKS ON ACTIVE INTERFACES 
networksetup -listallnetworkservices | grep -v asterisk | while read line ; do 
t=`networksetup -getinfo "$line" | grep "IP address" | grep -v none`
if [ "$t" ]; then
   tt=`networksetup -getsocksfirewallproxy "$line" | grep -v Authenticated | grep "Enabled: Yes"` 
   if [ "$tt" ]; then
      echo Disabling SOCKS on $line
      networksetup -setsocksfirewallproxystate "$line" off
   fi
fi
done 



Description of the key commands:
  • networksetup -listallnetworkservices
    Displays a list of all the network services on the server's hardware ports. An asterisk (*) denotes that a network service is disabled.
    Sample output:
             USB Ethernet
             AirPort
             Bluetooth DUN
  • networksetup -getinfo "Airport"
    Displays the IP address, subnet mask, router, and hardware address for the device that you specify.
  • networksetup -getsocksfirewallproxy Airport
    Displays SOCKS proxy (server, port, enabled value) info for the device.
    Sample output:
             Enabled: No
             Server: localhost
             Port: 1080
             Authenticated Proxy Enabled: 0
  • networksetup -setsocksfirewallproxy Airport localhost 1080
    Set and enable the SOCKS proxy server  localhost:1080.
  •  ssh -C -D *:1080 your.remote.host
    Launch ssh with dynamic port forwarding
  • networksetup -setsocksfirewallproxystate Airport off
    Disables SOCKS proxy.

Sources:

Wednesday, November 23, 2011

SSH Dynamic Port Forwarding (SOCKS server)

Dynamic port forwarding turns your SSH client into a SOCKS proxy server. SOCKS is widely-implemented protocol for connecting programs to Internet through a proxy server. Each program that uses the proxy server needs to be configured specifically.
Say you wanted Firefox to connect to every web page through your SSH server in a remote host.

First you would setup the dynamic port forwarding with the default SOCKS port:

ssh -C -D *:1080 your.remote.host

The -D option specifies dynamic port forwarding. 1080 is the standard SOCKS port, and *: indicates to listen in all the interfaces (not only 127.0.0.1) . -C enables compression, which speeds the tunnel up.

Next you would tell Firefox to use your proxy:
  • go to Edit -> Preferences -> Advanced -> Network -> Connection -> Settings...
  • check "Manual proxy configuration"
  • make sure "Use this proxy server for all protocols" is cleared
  • clear "HTTP Proxy", "SSL Proxy", "FTP Proxy", and "Gopher Proxy" fields
  • enter "127.0.0.1" for "SOCKS Host"
  • enter "1080" for Port.
You can also set Firefox to use the DNS through that proxy, so even your DNS lookups are secure:
Type in about:config in the Firefox address bar
Find the key called "network.proxy.socks_remote_dns" and set it to true

To make other programs use your SSH proxy server, you will need to configure each program in a similar way.
The SOCKS proxy will stop working when you close your SSH session.

Source: https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding#Dynamic_Port_Forwarding

Saturday, October 1, 2011

Restart mDNSResponder

The mDNSResponder process resolves the names, but sometimes when changing the location gets stuck.
A quick fix is to restart it, and hope it will work again:
sudo launchctl unload /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
sudo launchctl load /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist

Other resources:
http://support.apple.com/kb/HT3789
http://support.apple.com/kb/HT4030
http://reviews.cnet.com/8301-13727_7-10471471-263.html
https://discussions.apple.com/thread/2227251?threadID=2227251

Monday, June 20, 2011

Configure SSH jump hosts

Here is a configuration for establishing an ssh connection to a hidden host via a jump host.
This allows to connect hiddenhost just by typing (also works with scp and sftp)
    > ssh hiddenhost
if public keys for both jump.host and 10.0.0.1 are available in the local host,
then the connection will be established without prompting for any password.
Edit: .ssh/config and add the entry:


# connect: username@hiddenhost (10.0.0.1) via me@jump.host
Host hiddenhost
   User username
   HostName 10.0.0.1
   ProxyCommand ssh  me@jump.host  nc %h %p
   ForwardAgent yes

Alternatively with the following line is possible to establish the connection to 10.0.0.1, 
but the local key will not be presented to 10.0.0.1

    > ssh -t me@jump.host   ssh  username@10.0.0.1

Source: http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts#Jump_Hosts_--_Passing_through_a_gateway_or_two

Sunday, June 12, 2011

Connecting to WiFi Network with hidden SSID

In order to connect to a network with hidden SSID (i.e. that does not broadcast it) in OS X 10.5 (in 10.6.8 this issue has been solved) the network must be configured each time we want to connect, even if the network appears in the advanced configuration preferences.

To the moment the best solution I've found is to forces the connection with the following shell command:
sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport --associate=yourHiddenSSID --password=yourkey

Source: http://superuser.com/questions/43836/automatically-connecting-to-hidden-ssid-wifi-network

Thursday, September 9, 2010

Building a chroot environment for sftp and sshd (OpenSSH)

This is useful when you don't trust your users, and still must provide sftp service.

The following script creates a chroot enviroment at the specified path.
It has been adapted to run on OSX. Use it with:
  > ./create_chroot_env_osx /tmp/testdirectory
  > sudo chroot /tmp/testdirectory


#!/bin/bash
#
# Usage: ./create_chroot_env_osx destinationPath
#

# Specify HERE the apps you want into the enviroment
# /usr/lib/dyld is mandatory
APPS="/usr/lib/dyld /bin/bash /bin/ls /bin/mkdir /bin/mv /bin/pwd /bin/rm"

# Sanity check
if [ "$1" = "" ] ; then
   echo "     Usage: $0 destinationPath"
   exit
fi

# Go to the destination directory
if [ ! -d $1 ]; then mkdir $1; fi
cd $1

# Create Directories no one will do it for you
mkdir etc
mkdir bin
mkdir usr
mkdir usr/bin
mkdir usr/lib
mkdir usr/lib/system

# Add some users to ./etc/paswd
grep /etc/passwd -e "^root" > etc/passwd
grep /etc/group -e "^root" > etc/group

# Copy the apps and the related libs
for prog in $APPS;  do
   cp $prog ./$prog

   # obtain a list of related libraryes
   ldd $prog > /dev/null
   if [ "$?" = 0 ] ; then
      LIBS=`ldd $prog | grep version | awk '{ print $1 }'`
      for l in $LIBS; do
         cp $l ./$l
         # second level of dependent libraryes
         LLIBS=`ldd $l | grep version | awk '{ print $1 }'`
         for ll in $LLIBS; do
            cp $ll ./$ll
         done
      done
   fi
done

Monday, March 8, 2010

Command line wifi

This is the command line interface for the wifi card (Airport).
A couple of useful commands: For scanning the available networks
sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -s
And, to query the status of the current connection
sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I


Source: http://osxdaily.com/2007/01/18/airport-the-little-known-command-line-wireless-utility/

Wednesday, December 2, 2009

MAC Address Spoofing


Change the MAC address to 00:01:02:03:04:05  in OSX:
sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport -z
sudo ifconfig en1 ether 00:01:02:03:04:05
The first command disassociate from the current network, and the second changes the MAC address. 
Verification:
ifconfig en1 | grep ether

Change the MAC address to 00:01:02:03:04:05  in Linux:
 sudo ifconfig eth0 down
 sudo ifconfig eth0 hw ether 00:01:02:03:04:05
 sudo ifconfig eth0 up



Tuesday, September 22, 2009

Mount OS X NFS share from Linux

The nfs server that comes with OS X is picky.



------------ IN THE SERVER OSX ------------
->/etc/exports
/Users/ 192.168.114.128
/Users/ localhost


->/etc/nfs.conf
nfs.server.require_resv_port = 0
nfs.server.mount.require_resv_port = 0

> sudo nfsd enable
> sudo showmount -e



---------FROM LINUX ---------

sudo mount 192.168.114.2:/Users /mnt/OSXUSERS

The user in the linux system MUST have the same UID as in the OSX, the mapall option doesn't work.

Creating an SFTP mount in Mac OS X (with Macfusion)

Source: