Browsed by
Author: Varal7

Today, I created my own marketplace with Stripe

Today, I created my own marketplace with Stripe

Today, I created my own marketplace with Stripe and it was fun!

As part of GitHub’s student developer pack, Stripe comes with a transaction fee waiver for the first 1000$ processed (or so I thought).

Now, I can use this marketplace to sell my stuff to my neighbors at École polytechnique… and they can pay me by credit card!

As a matter of fact it actually is easy to use Stripe’s API: it comes with snippets of code that you can directly use in your application. Even your sandbox keys are pre-filled.

I had a good experience using Trello’s API and using Stripe’s API gave similar satisfaction (much to the contrary of the RATP’s API).

First, encrypt!

The first obstacle I encountered was that Stripe obviously requires TLS encryption. Hopefully, Let’s Encrypt allows that super easily. Like you just install certbot and type certbot certonly and it does the work.

This was very rewarding. As you can see, this blog now uses HTTPS. I will also be able able to use my own server for building a Facebook Messenger Chatbot without using Heroku!

Then, configure the nginx server

I also had some trouble configuring the nginx server just like I wanted: with PHP, TLS and .htpassw. At the end, it looked like this:

server {
        listen 80;
        server_name sold.varal7.fr;
        return 301 https://$server_name$request_uri;
}

# HTTPS server

server {
    listen 443;
    root /var/www/sold;
    server_name sold.varal7.fr;
    index index.html index.htm index.php;
    client_max_body_size 10M;

    ssl on;
    ssl_certificate /etc/letsencrypt/live/sold.varal7.fr/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/sold.varal7.fr/privkey.pem;

    ssl_session_timeout 5m;

    ssl_protocols SSLv3 TLSv1;
    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
    ssl_prefer_server_ciphers on;

    location /admin/ {
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/.htpasswd;

        location ~ \.php {
             fastcgi_index index.php;
             fastcgi_pass unix:/var/run/php5-fpm.sock;
             fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
             include fastcgi_params;
         }
    }

    location ~ \.php {
         fastcgi_index index.php;
         fastcgi_pass unix:/var/run/php5-fpm.sock;
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         include fastcgi_params;
     }
}

(Yeah, sorry I only serve ipv4 for now…)

To generate the .htpsswd file, I simply typed sudo htpasswd -c /etc/nginx/.htpassw varal7 in terminal.

Finally, the coding part

I wanted to do produce the easiest possible code for my MVP. So I used vanilla PHP.

The code is available as a GitHub repository.

End of the day, I don’t think it was the best thing to do, as I had to recode a lot of useful (and therefore existing) functions. I could have used a simple framework like Silex.

Success! Less than 24 hours after starting the project, I had my first customers and will soon cash in some money 🙂

Sadly, as you can see, Stripe does seem to take a transaction fee anyway. I have wrote them an email about that.

Bonus: receiving SMS notification

Stripe doesn’t notify anyone automatically, so you have to do it yourself.

Sadly, I don’t have my own email server yet. But I already have a SMS server ready!

More precisely, I have a Raspberry Pi equipped with a SIM card, connected to the same VPN as my VPS and listening on a certain port for a GET request. If it gets the correct one, it sends me a text with a certain message. I guess this setup could be the subject for another article and it will, when I feel like writing it.

Finally, adding this SMS notification was not difficult and I now have this feature in production 🙂

How I “hacked” into RATP’s API

How I “hacked” into RATP’s API

Jan, 3rd 2017: La RATP (the company in charge of Paris’s underground train network) has finally opened its API for real-time traffic.
Jan, 5th 2017: I email my subscription form in order to be able to use it.
Jan, 17th 2017, 2pm: RATP’s Open Data team eventually answers me and my IP is allowed access to the API.
Jan, 19th 2017, 2am: Finally! After hours of going crazy over the technology used by WSIV (Web Service Information Voyageur), after a few giving up/getting back to it moments, it works. I am now able to display every possible information about the next trains going from the École polytechnique train station (Lozère) to Paris.

Here is the code I ended up with:

<?php
$request = '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://wsiv.ratp.fr/xsd" xmlns:wsiv="http://wsiv.ratp.fr">
    <soapenv:Header/>
 <soapenv:Body>
     <wsiv:getMissionsNext>
        <wsiv:station>
            <xsd:line>
                <xsd:id>RB</xsd:id>
            </xsd:line>
            <xsd:name>Lozere</xsd:name>
        </wsiv:station>
        <wsiv:direction>
            <xsd:sens>A</xsd:sens>
        </wsiv:direction>
     </wsiv:getMissionsNext>
 </soapenv:Body>
</soapenv:Envelope>';
$location = "http://opendata-tr.ratp.fr/wsiv/services/Wsiv?wsdl=";
$uri      = "http://opendata-tr.ratp.fr/wsiv/services";
$action   = "urn:getMissionsNext";
$version  = 0;

$client = new SoapClient(null, array('location' => $location,
                                    'uri'       => ""));
$xmlstring = $client->__doRequest($request, $location, $action, $version);
$clean_xml = str_ireplace(['SOAPENV:', 'NS1:', 'NS2:'], '', $xmlstring);
$xml       = simplexml_load_string($clean_xml);

$return        = $xml->Body->getMissionsNextResponse->return;
$perturbations = array();
$missions      = array();

$station     = $return->argumentStation->name;
$direction   = $return->argumentDirection->name;
$line        = $return->argumentLine->reseau->name . ' ' . $return->argumentLine->code;

echo "Station $station ($line) en direction de $direction",PHP_EOL;

foreach($return->perturbations as $perturbation) {
    echo $perturbation->message->text,PHP_EOL;
}

foreach($return->missions as $mission) {
    $id = isset($mission->id) ? $mission->id . ' ' : "";
    echo $id, $mission->stations[1]->name, ': ', $mission->stationsMessages;
}

I needed to use PHP for this implementation because one goal was to include this snippet into Frankiz, the school’s students (old) website.

Two days ago, I had no idea what a Web Service was, nor what WSDL stood for, and I thought SOAP was… well you know… soap.

Looking into the API’s documentation

In theory, everything I needed can be found on RATP’s website, where I was able to download the developer’s kit, a .zip file, with the following content :

├── CG-API-RATP.pdf
 ├── FO-inscription.pdf
 ├── ratp-wsiv-opendata
 │   ├── Wsiv.wsdl
 │   ├── doc_wsiv.html
 │   ├── exemple.pdf
 │   ├── index.html
 │   └── ratp.gif

Fast-forward the administrative procedure for allowing my VPS’s IP to request their API, I was thrilled to open the exemple.pdf file.

The examples demonstrated that this API exposed what we would expect from it: lines, stations, timestamps of the next passages. However, the examples were simply unexploitable: instead of raw text, the request was given as an image where we can not even see the full text.

So, back to to the doc_wsiv.html file, the API’s official documentation. Sadly, this is not the documentation you would be used to (especially if you have ever tasted/tested Trello’s API).

I applied my usual strategy an launched up Postman.

Understanding WSDL and SOAP

Postman is the must-have tool when it comes to web requests. I use it to replay API calls, amongst other things.

Eventually, I found this page from Postman’s documentation explaining how I could use Postman to make SOAP requests, too.

The idea is: you simply make a POST request using raw text/xml content, which you define to be the content of your SOAP request.

It worked quite well with the Holiday Service example, so I was quite optimistic.

Using Postman to hit the RATP API

At first, it was quite unconvenient to carry out tests on the RATP API because only the IP of my VPS was allowed to initiate the connection. Hopefully, using my VPN, I was able to send requests to the RATP servers directly from my computer but with the correct IP.

At first, I naively thought that I would hit something by recycling my request from the Holiday Service example, simple changing the xmlns:hs attribute and adapting the URL.

But what URL? Reading several times through the API’s documentation, I did not find any mention of it. It was time to open the Wsiv.wsdl file. Here, I found a few candidates : http://wsiv.ratp.fr/ and http://www.ratp.fr/wsiv/services/Wsiv?wsdl did not work. But I eventually found http://opendata-tr.ratp.fr/wsiv/services/Wsiv, which DID return an error. Hourray.

 

Once I understood what URL I was supposed to request at, I was far from being done. I chose the POST method, the Headers needed to be custom defined: Content-Type=text/xml. In the Body tab, I chose the raw method and indicated that I used XML.

Here are a few error messages I got and how I managed to handle them.

<faultstring>The endpoint reference (EPR) for the Operation not found is http://opendata-tr.ratp.fr/wsiv/services/Wsiv and the WSA Action = null</faultstring>

Solution (from StackOverFlow): Add the header SOAPAction to the request. Adding a wrong action will actually improve the situation but the correct one needs to be determined by reading the .wsdl file.

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
        <soapenv:Fault>
            <faultcode>soapenv:Server</faultcode>
            <faultstring>unknown</faultstring>
            <detail />
        </soapenv:Fault>
    </soapenv:Body>
</soapenv:Envelope>

Solution: Did not find at the time. This one drove me crazy. The error is unknown. What I wanted was to call the action getLines without parameters (which is the easiest call possible), but that just did not work.
At that point, I eventually managed to display the list of all lines… when I stopped trying to display all lines and display only. My mistake was think my request was correct. It made me lose a lot of time to think that I understood something when I really didn’t.

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
        <soapenv:Fault>
            <faultcode>soapenv:Server</faultcode>
            <faultstring>com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "wsiv"
 at [row,col {unknown-source}]: [5,14]</faultstring>
            <detail />
        </soapenv:Fault>
    </soapenv:Body>
</soapenv:Envelope>

Solution: It took me time to understand why and how I was supposed to declare a namespace. In RATP’s documentation examples, it happens that the first line is not totally shown and that is where the namespaces are actually defined. Drove me crazy as well, but forced me to learn about namespace. By trial-and-error, I eventually figured out that i was supposed to add the attribute xmlns:wsiv and xmlns:xsd.

In the end, it was both frustrating and fun to play around with this API. Frustrating because the documentation is very obscure for someone knows nothing about Web services, fun because I like challenges and there were a lot of “Tada” moments.

Here are some requests that probably made me “Tada”.

Jan 17: Getting the API to display lines

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
 <soapenv:Body>
 <wsiv:getLines>
 <xsd:id>RA</xsd:id>
 </wsiv:getLines>
 </soapenv:Body>
</soapenv:Envelope>

Sadly, it does not do as expected (that is, to display info on line RA). And no, it does not work, if I remove the “RA” line.

Jan 18: Getting the API to display lines with a request that actually makes sense.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://wsiv.ratp.fr/xsd" xmlns:ns1="http://wsiv.ratp.fr">
 <soapenv:Body>
 <wsiv:getLines>
 <xsd:blblblblbl>
 </xsd:blblblblbl>
 </wsiv:getLines>
 </soapenv:Body>
</soapenv:Envelope>

Okay, to be honest, there were initially not that many “bl”, but yes, that would work.

Jan 18: Getting the API to display info about a given line

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://wsiv.ratp.fr/xsd" xmlns:wsiv="http://wsiv.ratp.fr">
<soapenv:Header/>
 <soapenv:Body>
 <wsiv:getLines>
 <wsiv:line>
 <xsd:id>RB</xsd:id>
 </wsiv:line>
 </wsiv:getLines>
 </soapenv:Body>
</soapenv:Envelope>

Yes! I was finally able to give the name of a line, to finally give a variable to the API! The next steps was to fetch info about a given station, and that was easy enough.

The body of the request I really cared about was in the end:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://wsiv.ratp.fr/xsd" xmlns:wsiv="http://wsiv.ratp.fr">
<soapenv:Header/>
 <soapenv:Body>
 <wsiv:getMissionsNext>
 <wsiv:station>
 <xsd:line>
 <xsd:id>RB</xsd:id>
 </xsd:line>
 <xsd:name>Lozere</xsd:name>
 </wsiv:station>
 <wsiv:direction>
 <xsd:sens>R</xsd:sens>
 </wsiv:direction>
 </wsiv:getMissionsNext>
 </soapenv:Body>
</soapenv:Envelope>

With the header SOAPAction=urn:getMissionsNext.

Using CURL to make the request using PHP

Postman has this nice little feature that allows you to generate the curl code for PHP: simply click on the arrow next to the “Save” button.

But the party was far from being over.

Parse the XML

At that point, I thought I was almost done. It only remained to parse the XML. However, SimpleXML would not work out of the box and the XML Object kept being Empty.

Encouraged by this StackOverFlow‘s second answer, I tried to replace everything I had done with PHP5’s SoapClient.

Making the Request using SoapClient

It was even worse. I read the whole SoapClient documentation twice. I played with all kind of combinations of SoapParam and SoapVar. I tried to rewrite my own __doRequest() and __soapCall() methods. I read the C code of this function PHP5’s GitHub repository.

This is where what I call the hacking took place. I am not a hacker, but my friends who are, they tell me that it’s a lot a trial-and-error.

But in the end, nothing seemed to work. I was so frustrated. Good point is, in the process, I understood a lot more about how SOAP worked.

The final hack

What I ignored is the hack suggested by this StackOverFlow‘s first answer.
A one-liner that just does it. Such a beautiful hack.

 

UPDATE: By comparing the results, I figured out that CityMapper already used this API. Not sure when, though? My implementation can now be found on a GitHub repository and a live demo is available here.

How to create a chatbot for your Facebook page using Heroku in 10 minutes

How to create a chatbot for your Facebook page using Heroku in 10 minutes

It is now a very popular thing for organizations to own a “Page” on Facebook. Users are now used to communicate directly with a Facebook Page and they expect a quick answer to their questions, 24/7.

image0

Hopefully, since April 2016, Facebook allows us to create messenger bots.
Here’s how.

Note that everything that follows can be found on Facebook’s documentation.

First, create a Facebook Page

This is an easy step. Simply follow the instructions here: Create a page

Then, create a Facebook App

This is another easy step. Follow this link: Create an app
Facebook will want to help you set up your app with “Quick start” steps. Just skip this.

Take note of your App Secret.

On you app Dashboard, under Product Settings, click “Add Product” and select “Messenger.”

image1

Under “Token Generation”, choose your Page to generate a Token. Take note of that Page Access Token.

And keep this tab open. You will need it soon.

Deploy to Facebook’s messenger-platform-samples to Heroku

Because Facebook’API needs an HTTPS callback, you won’t be able to test your app from localhost. One easy way to test your app is to use Heroku.

  1. Create an account on Heroku.
  2. Now download Heroku-CLI from Heroku’sdocumentation
  3. Let’s go

git clone https://github.com/fbsamples/messenger-platform-samples
cd messenger-platform-samples/node
heroku login
heroku create youroriginalnamehere #Replace with your original name
cp config/default.json config/production.json

Now edit the config/production.json file with the App Secret and the Page Access Token you got from last step.

The serverURL is something that should look like https://youroriginalnamehere.herokuapp.com/

Now, you get to choose your Validation Token. Take note of it, you will need it in a moment.

git add .
git commit -m "Prod"
git push heroku master

Add a Messenger webhook to your app

Back to your App Dashboard. In the Webhooks section, click “Setup Webhooks.”

The URL for your webhook is something like https://youroriginalnamehere.herokuapp.com/webhook.

Enter your Validation Token and select message_deliveries, messages, messaging_optins, and messaging_postbacks under Subscription Fields.

You can now subscribe your App to your Page.

Tada

You’re done!

You can now try to chat with your brand new Facebook bot. By default, it will simply echo your message. In development mode, it will only be accessible to admins of the page and testers of the app. Enjoy!

How Technology Will Help You Reach Your Goals

How Technology Will Help You Reach Your Goals

Last Sunday, I ran the Paris-Versailles. The Paris-Versailles is a 16km race. You start from Paris, in front of the Eiffel Tower and you arrive at the Palace of Versailles. It is known to be quite difficult especially because of the altitude difference between kilometers 6 and 8.

denivelle-eng2015

Now, I am not a a very good runner. And I think I would not have managed to run through all this race without Spotify.

img_1485-2_iphone6_gold_portrait

Spotify has a feature that helps you keep a constant pace when running. You set it to certain tempo, say 155 beats per minute, and the music will automatically adapt to match this tempo. Even the transitions between the tracks are smooth and respect the tempo.

Now imagine yourself, running along the Seine river, the music beats in sync with every step you take. You breathe with your mouth, inhale every fourth beat, exhale during the next three. More than twice every second, each time your foot touches the ground, the bass resonates into your ear, and your body generates dopamine to reward you for being in sync with the music.

img_1459_iphone6_gold_portrait

Yes, that is how I ran the Paris-Versailles.

I believe technology should allow us to reach our goals. It should help us socialize, meet other people instead of imprison us into Facebook and text messages. It should motivate us to go out and exercise instead of make us stay home, playing FIFA or Candy Crush. It should take us out of our comfort zone and make us grow, instead of turning us into some lazy piece pancake.

I dream of a world where technology actually make us happier. Let’s make it happen.

Why every sysadmin should know about nmap

Why every sysadmin should know about nmap

The other day, one friend texted me in panick because her computer could not connect to my school’s local network anymore. Until recently, when we set up a DHCP server for them, my school relied on manual IP addresses (with hardware filtering). But in some specific places in the campus, we still use manual IP addresses, without any filtering. And sometimes, like in the present case, people steal each other’s IP.

Well, it’s not really stealing. People just don’t know which IP is in use.

My first guess was to use Wireshark, in order to listen to the packets that were going to and from this IP, and identify the corresponding hardware address.

But there is in an easier way to get information about the computer using an IP address.

sudo nmap -O 192.168.1.1

Which yields the following result:

Starting Nmap 7.01 ( https://nmap.org ) at 2016-10-05 16:13 CEST
Nmap scan report for 192.168.1.1
Host is up (0.034s latency).
Not shown: 997 closed ports
PORT STATE SERVICE
22/tcp open ssh
53/tcp open domain
80/tcp open http
MAC Address: 30:B5:C2:70:50:72 (Tp-link Technologies)
Device type: WAP
Running: Linux 3.X
OS CPE: cpe:/o:linux:linux_kernel:3.10
OS details: OpenWrt Barrier Breaker (Linux 3.10)
Network Distance: 1 hop

The option -O performs a OS-detection.
And if your target is on the same network as you, then nmap will tell you its network address.
Easy as pie 🙂

What I Learned After One Year of Productivity-tweaking

What I Learned After One Year of Productivity-tweaking

Getting work done is difficult. I am a geek with computer skills, and I am no exception: I am lazy as f*.

If you want to be a good programmer, you need to think lazy: what is the best way to create the maximum value with minimum effort?

And this is where my productivity journey started. Everyday I wake up with this question in my mind: how do I make best usage of this beautiful day?

Now, I am giving you my three simplest tips to get you started on this amazing journey. Three tips that you can try starting TODAY.

As a disclaimer, those tips work for me. Adapt them so they suit you.

1. If it takes less than 2 minutes, just do it now

I owe this trick to David Allen’s book Getting Things Done.

During your usual day, you will get tons of inputs: emails, text messages, Facebook messages, phone calls, a talk in the corridor or just random thoughts.

Obviously, you should collect all this data into one place. But when this input requires a task and that task can be done in less than 2 minutes, then just do it now!

Let me emphasize: if it can be done in less than 2 minutes, do it now.

When it takes more time and energy to store it than to get it done, only a fool would not do it!

(But it comes to productivity, we are all fools, that’s true…)

2. Plan it before you start it

Give me six hours to chop down a tree and I will spend the first four sharpening the axe. – Abrahm Lincoln

In the same manner as preparation is the key to success, planning is the key to getting things done.

To all the people who do not plan their day, please try this: Plan you day!

To all the people who plan their day in the morning, please try this: Plan you day just the day before, plan you week just before it starts.

In the evening, just before you go to sleep, your mind is not ready to do anything but can be in anticipation of what comes next. In the morning, you should not lose time deciding what to do: you should take the first item of your to-do list and just do it.

3. Make regular breaks

Work expands so as to fill the time available for its completion. – Parkinson

When you have work to do, split it in smaller tasks as much as possible. Then timebox yourself on each of these tasks. Start a timer and get to work. And the end of the timebox, switch to another task.

Humans tend to easily lose focus. I personally know that if a task takes me more than 40 minutes, I won’t be 100% concentrated.

Try this: give yourself 40 minutes for a task and try to be as focused as possible. Mute any devices that might disturb you. At the end, take a break and relax. Give a quick look to emails and messages for any urgent stuff. Then repeat.

You should also allocate a 40-minutes slot for email-handling.

This tip also applies in a more macro scale: everyday, after lunch, I go for a 20-minutes power-nap. This breaks my day in half and allows me to feel far more relaxed and focused in the afternoon.

Finally

Those are just three productivity habits that you can apply right now to start getting things done. You should apply them everyday during one month to build them into habits. Good luck for your productivity journey!

The Secret of Getting Ahead…

The Secret of Getting Ahead…

 

One of the most difficult things in life is setting your own goals. Whether you want to become a billionaire, to live on Mars, or to figure out the cure for cancer, what matters most is to have a strongly conscience of why you do what you do: to achieve this goal.

It is not who you are that matters, it is what you want to be. Because what you want to be defines what you do. And what you do defines you.

After one year of reading books, watching inspirational speeches, going through hundreds of blog posts about productivity and reaching goals, and trying hacks and tweaks, here is the most important quote I remember.

The secret of getting ahead is getting started. – Mark Twain

You define your goal, then you just start working towards it. Today. Period.