Posted on

DYI NAS

This a tutorial for a “Do it your self ” NAS Bay using a Raspberry Pi 3 Model B thanks to techradar for the steps to follow, to enhance the tutorial I wanted to make a video explaining what to do. http://www.techradar.com/how-to/computing/how-to-build-your-own-raspberry-pi-nas-1315968

 

Posted on

Editorial

Hi everyone,

Next 11 November I’ll be at the Hackathon Datascapes organized by the McGill Innovation Week. I’ll tell you about the challenges that we’ll be facing with other developers. This Hackathon is about Montreal city data to help develop decision making tools and improve city services.

Larbi 😉

Posted on

Datascapes 2018 data visualisation

During this last Saturday I had the opportunity to discover the word of data analysis and visualisation.

Data visualisation

Font

Font-type: sans-serif serif
Font size
Fype face / weight: light italics bold
Emphasis: font Font Font

Fewer font types and faces are better
Choose font accordingly

Color

Color contrast
Color theory (use websites that can help with colors)
Fewer colors, use selection helpers
http://w3chools.com/colors (Color schemes)
colorbrewer2.org

Symbologie
Graphical semiotics by Bertin 1974

Charts
Bars
Histograms
Heatmaps
Density functions
If it doesn’t add any value than don’t do it (2d vs 3d excel charts)

Classifications
Quantiles classification (hardest)

User helpers in your charts (User better legend)

Programming tools

R language and R studio are powerful tools to quickly display data into charts and filter data to display big data sets

ggplot2 in R

D3.js

Story maps (visualizing the context)

Here are the data sets that we where using the Hackathon most of them are from the city of Montreal open data website GitHub: https://github.com/CSCDS/datascapes-2017

http://donnees.ville.montreal.qc.ca/

Posted on

How to install Django on a Hostgator shared hosting

Thanks to that help full document that I found on a Forum (I can’t )

https://docs.google.com/document/d/1jhvixMmTRGYHRbHaYlTHTNrtpziGotQMc0iBO0sTbIo/edit

Here is a step by step tutorial to install Django framework on your shared hosting. But first of all it is important to be aware that the performance is not the best on a shared Hosting. This solution is a good one for a small scaled solutions, on for a website that doesn’t have a lot of visitors. Otherwise it is better to purchase a dedicated server for a better performance and experience for your application users. Also you are limited in term of Python Modules the Hostgator list above show what you have access to (http://support.hostgator.com/articles/hosting-guide/hardware-software/python-modules) according to this forum post : http://grokbase.com/t/gg/django-users/1313jnz7hv/django-installation-on-hostgator

 

1- First of all you need to enable your access via SSH to your server. The best way to do this (and the only way I found) is by contacting via the chat (chat.hostgator.com) or phone the customer service.

2 – Once you access is enabled connect to your server via SSH (Windows users can access trough Putty)

For Linux / Mac users type the following command in the Terminal:

ssh -p 2222 cpanelusr@ip.add.re.ss

For Windows Putty users :

Hotsname / Server IP : yourmaindomain.com / or the server IP (you can find this information on your Cpanel on the right sidebar in the general information section)

Username and Password : Use your Cpanel Credentials

Port : 2222 (for shared hosting otherwise it’s 22 for dedicated servers according to Hostgator help page http://support.hostgator.com/articles/how-do-i-get-and-use-ssh-access)

The following steps are from the source document linked above :

 

(this probably out of date…)

How to install Django on a basic shared hosting plan with plain CGI

I’ve successfully installed Django on my Hostgator account (‘Baby Croc’ plan) using these steps. Hostgator has been really great for me, but they don’t support fastCGI. Using vanilla CGI with Django is not generally recommended and results in crappy performance, but for a low traffic site it’s bearable (barely).

 

Here are some links to source material for this document:

 

http://www.djangoproject.com/documentation/install/

http://code.djangoproject.com/ticket/2407

http://mysql-python.sourceforge.net/

http://www.python.org/dev/peps/pep-0333/

http://www.djangoproject.com/documentation/modpython/#serving-the-admin-files

 

Prerequisites

 

SSH shell access to your account

Python 2.3+ (Python 2.4 preferred)

MySQLdb module (ask your support person to install this)

Some knowledge of Django and Python

 

Install django and create a project directory

 

Create a directory in your home directory called “django” (or whatever) that is outside of your document root.

 

mkdir ~/django

cd ~/django

svn co http://code.djangoproject.com/svn/django/trunk/django

 

Create a directory for your Django projects.

 

mkdir ~/django/projects

 

You should now have the following folders:

 

~/django/django

~/django/projects

 

Edit your .bash_profile and add the following:

 

export PATH=”.:/home/username/django/django/bin:$PATH”

export PYTHONPATH=/home/username/django

Then enable the changes:

source .bash_profile

 

Add the following script to your cgi-bin directory and name it wcgi.py

 

#!/usr/local/bin/python

 

import os, sys

 

sys.path.insert(0, “/home/username/django/”)

sys.path.insert(0, “/home/username/django/projects”)

sys.path.insert(0, “/home/username/django/projects/newprojects”)

 

import django.core.handlers.wsgi

 

os.chdir(“/home/username/django/projects/newproject”)    # optional

os.environ[‘DJANGO_SETTINGS_MODULE’] = “newproject.settings”

 

def runcgi():

    environ                      = dict(os.environ.items())

    environ[‘wsgi.input’]        = sys.stdin

    environ[‘wsgi.errors’]       = sys.stderr

    environ[‘wsgi.version’]      = (1,0)

    environ[‘wsgi.multithread’]  = False

    environ[‘wsgi.multiprocess’] = True

    environ[‘wsgi.run_once’]     = True

 

    application = django.core.handlers.wsgi.WSGIHandler()

   

    if environ.get(‘HTTPS’,’off’) in (‘on’,’1′):

        environ[‘wsgi.url_scheme’] = ‘https’

    else:

        environ[‘wsgi.url_scheme’] = ‘http’

   

    headers_set  = []

    headers_sent = []

   

    def write(data):

        if not headers_set:

             raise AssertionError(“write() before start_response()”)

       

        elif not headers_sent:

             # Before the first output, send the stored headers

             status, response_headers = headers_sent[:] = headers_set

             sys.stdout.write(‘Status: %s\r\n’ % status)

             for header in response_headers:

                 sys.stdout.write(‘%s: %s\r\n’ % header)

             sys.stdout.write(‘\r\n’)

       

        sys.stdout.write(data)

        sys.stdout.flush()

   

    def start_response(status,response_headers,exc_info=None):

        if exc_info:

            try:

                if headers_sent:

                    # Re-raise original exception if headers sent

                    raise exc_info[0], exc_info[1], exc_info[2]

            finally:

                exc_info = None     # avoid dangling circular ref

        elif headers_set:

            raise AssertionError(“Headers already set!”)

       

        headers_set[:] = [status,response_headers]

        return write

   

    result = application(environ, start_response)

    try:

        for data in result:

            if data:    # don’t send headers until body appears

                write(data)

        if not headers_sent:

            write(”)   # send headers now if body was empty

    finally:

        if hasattr(result,’close’):

            result.close()

 

runcgi()

 

Edit your root .htaccess file and add the following lines

 

RewriteEngine on

RewriteRule ^media – [L]

RewriteRule ^cgi-bin – [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteRule ^(.*)$ /cgi-bin/wcgi.py/$1 [QSA,L]

 

Create a new django project

 

(or upload your existing project)

It might be a good idea to create a new dummy project just to have a baseline to test your installation.

 

cd ~/django/projects

django-admin.py startproject newproject

cd newproject

manage.py startapp newapp

 

Try it out

 

Since DEBUG is True by default (in settings.py) you should get the default Django 404 page.

Make sure you set DEBUG=False before you open your site to the public.

 

Notes:

 

The admin media files reside in django/contrib/admin/media and since they aren’t accessible from your document root you need to create a symbolic link to the media files (or just copy them):

 

ln -s ~/django/django/contrib/admin/media ~/www/media

 

Where “~/www/” is your document root.

Posted on

How to connect an external microphone to your Smartphone

I always wondered why my 10$ external lapel microphone I bought for my GOPRO won’t work with my smartphone. The answer is that the ground and the Mic connectors are inverted between a microphone 3.5 mm male jack and a smartphone female 3.5 mm jack connector. This video shows step by step a diy (do it yourself) adapter to connect any microphone with a 3.5 mm jack to your smartphone. If you are too lazy you can still buy the MyMyk adapter for about 30 $.

List of tools :

  • Lapel microphone: http://www.ebay.ca/itm/Deluxe-Lavalier-Lapel-Microphone-Clip-on-Omnidirectional-Condenser-Mic-for-Apple-/152202586564?hash=item236ffb29c4:g:ojMAAOSwU-pXrR-X
  • MyMyk adapter: http://www.ebay.ca/itm/MyMyk-Connector-Cable-For-External-Mic-to-be-used-with-Smart-Phone-Tablet-/131836441011?hash=item1eb21071b3:g:gMYAAOSwQupXUUO6
  • 3.5 mm Jack Femal Socket: http://www.digikey.ca/product-detail/en/cui-inc/SJ-43514-SMT-TR/CP-43514SJDKR-ND/1819833
Posted on

How to download a video from a protected website

You often want to save a video on your hard drive to be able to watch later on a travel when you don’t have access to internet. There is one way to do it on websites that uses HTTP Live Streaming HLS.

Update :

Quick method :

An easier method is to use directly the m3u8 file available by filtering “m3u8” in the developer Filter box.

http://c.brightcove.com/services/mobile/streaming/index/rendition.m3u8?assetId=5300522320001&pubId=618566855001&videoId=5300467502001

Replace the m3u8 url in the following ffmpeg command :

ffmpeg -i "url" -c copy -bsf:a aac_adtstoasc 11.mp4

Source : https://gist.github.com/Pusnow/2ec4612adad7898b4c21e76b7a5d1915

Long method :

Let’s take for example an episode of “LES RECETTES POMPETTES” http://noovo.ca/videos/les-recettes-pompettes/alex-nevsky when using the developper option (F12) on the Network section we can quickly see that small 10 seconds transport stream video files are received from the streaming server http://v10-a.akamaihd.net/hls/618566855001/201701/5300522320001/618566855001_5300522320001_s-4.ts?pubId=618566855001&videoId=5300467502001noovo

 

If we analyse the link we see the extension .ts, that is the 10 seconds transport stream. The file name ends with 4 that’s the 4th 10 second transport stream.

 

http://v10-a.akamaihd.net/hls/618566855001/201701/5300522320001/618566855001_5300522320001_s-4.ts?pubId=618566855001&videoId=5300467502001

 

If we look at the video duration on the website we see that it lasts 21 min we can than calculate the number of files to download 24*60/10 = 144 files.

With a small bash file we can loop trough the list of files and download them :


#!bin/sh

#Download transport stream files from server

server="http://v10-a.akamaihd.net/hls/618566855001/201701/5300522320001/618566855001_5300522320001_s-"

extension=".ts"

max=144

for i in `seq 1 $max` do wget "$server$i$extension" done

 

Last step and not the least is to concatenate all the files to output them into a single transport stream of 21 minutes. For that task we can use ffmpeg concatenate function (you can get ffmepg by using sudo apt-get ffmpeg command ) :

ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.ts

or inline loop the current folder .ts files using :

ffmpeg -f concat -safe 0 -i <(for f in ./*.ts; do echo "file '$PWD/$f'"; done) -c copy output.ts

The -sage 0 above is not required if the paths are relative.

The -i mylist.txt is a file text that contains the list of input files to concatenate it should be with the following format :


# this is a comment

file '/path/to/file1'

file '/path/to/file2'

file '/path/to/file3'

Finally the output.ts will be the name of your final file you should specify the output type .ts or another format .mkv

Source 1: https://fredericbournival.com/blog/comment-telecharger-les-videos-de-noovo-ca-vtele/

Source 2: https://trac.ffmpeg.org/wiki/Concatenate