Stack Data StructureMy own stack data structureJekyll2014-07-16T23:29:54+01:00http://hugocaracol.github.io/Hugo Santoshttp://hugocaracol.github.io/hugonsantos@gmail.comhttp://hugocaracol.github.io/career/career-outside-matrix2014-02-13T00:00:00+00:002014-02-13T00:00:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Every year student grads enter the work force. At least they try to.
Through job ads they hope to catch a glimpse of something that can take them out of present despair.
If lucky, they land a job in a multinational company.</p>
<p>The years go by and they’re still at the same place doing the same thing they did some years ago.
<em>At least I’m financially stable</em> - they think. But as this thought comes to mind, another one comes up…</p>
<blockquote>
<p>What you know you can’t explain, but you feel it. You’ve felt it your entire life, that there’s something
wrong with the world. You don’t know what it is, but it’s there, like a splinter in your mind, driving you mad.</p>
<p>–<cite>Morpheus</cite></p>
</blockquote>
<p>Every night they go home and browse the internet for some possible solution, feel sleepy, reboot
(dreaming about a different life)…</p>
<p>Until one day, someone shows up and offer them to choose between a blue and a red pill. </p>
<blockquote>
<p>You take the blue pill - the story ends, you wake up in your bed and believe whatever you want to believe.
You take the red pill - you stay in Wonderland and I show you how deep the rabbit-hole goes.</p>
<p>–<cite>Morpheus</cite></p>
</blockquote>
<p>This rabbit-hole is full of uncertainty, adrenaline and, in the end, a sense of purpose.</p>
<p>With their eyes open, they start to meet people like them, people they didn’t know that existed. Companies looking
forward to hire someone like them. People that challenge the <em>status quo</em> every-single-day. But… a <strong>BIG LOUD</strong> but…</p>
<blockquote>
<p>I’m trying to free your mind, Neo. But I can only show you the door. You’re the one that has to walk through it.</p>
<p>–<cite>Morpheus</cite></p>
</blockquote>
<p>Huge doubts in their minds. <em>Am I good enough? Will I let the people down?</em></p>
<blockquote>
<p>What are you waiting for? You’re faster than this. <strong>Don’t think you are, know you are</strong>. Come on. Stop trying to hit me and hit me.</p>
<p>–<cite>Morpheus</cite></p>
</blockquote>
<p><img src="/images/the_matrix_neo.jpg" alt="Neo" /></p>
<h4 id="go-hit-morpheus-not-me--">Go hit Morpheus, not me :-)</h4>
<p><a href="http://hugocaracol.github.io/career/career-outside-matrix/">Career outside the Matrix</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 13, 2014.</p>http://hugocaracol.github.io/data-mining/relevant-keywords-using-tf-idf2013-07-16T16:26:00+01:002013-07-16T16:26:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>TF-IDF is a numerical statistic that tells us how important some word is in a document. It stands for <strong>term frequency - inverse document frequency</strong>.
There are a lot of words with a high term frequency like <em>the</em>, <em>a</em>, <em>and</em>, but they, themselves, don’t convey much
meaning. In other words, they are not <strong>keywords</strong>.</p>
<!-- more -->
<p>The second part of this statistic is the <strong>inverse document frequency</strong> which tells us how common a word is across all documents. It
is calculated by dividing the total number of documents in a collection by the number of documents containing the specific word we want to analyse.
After that quotient is determined, we take the logarithm of that.</p>
<p><strong>TF-IDF</strong> = n_count_word_document <strong>x</strong> log2(count_all_documents / count_documents_containing_word)</p>
<p>Let’s get some data for some keywords in this blog post. </p>
<p><em>The values in the *TF</em> column include title of this post and exclude anything below this paragraph.
The values in the <em>No docs containing term</em> column result from googling the term we are analysing. It is assumed, for demonstration
purposes that google contains 50 billion documents indexed.*</p>
<table>
<tr>
<td><bold>Term</bold></td>
<td>TF</td>
<td>No. docs containing term</td>
<td>TF-IDF</td>
</tr>
<tr>
<td>tf-idf</td>
<td>3</td>
<td><center>0.662 B</center></td>
<td>18.72</td>
</tr>
<tr>
<td>statistic</td>
<td>2</td>
<td><center>0.0604 B</center></td>
<td>19.39</td>
</tr>
<tr>
<td>keyword</td>
<td>3</td>
<td><center>0.992 B</center></td>
<td>16.97</td>
</tr>
<tr>
<td>frequency</td>
<td>4</td>
<td><center>0.391 B</center></td>
<td>28</td>
</tr>
<tr>
<td>blog</td>
<td>1</td>
<td><center>9.34 B</center></td>
<td>2.42</td>
</tr>
</table>
<p>As you can see, the term <em>blog</em> does not make a good keyword. All other terms are a good fit.</p>
<p><a href="http://hugocaracol.github.io/data-mining/relevant-keywords-using-tf-idf/">Relevant keywords using TF-IDF</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on July 16, 2013.</p>http://hugocaracol.github.io/code/rtfm-extended2013-06-08T12:24:00+01:002013-06-08T12:24:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>As you know RTFM stands for Read the Fucking Manual. Today, I would like to introduce you to RTWFM which stands for <strong>Read the Whole Fucking Manual</strong>.</p>
<p>A lot of times when I thing something <em>should</em> work but doesn’t, I use google to find the answer. But google as a <em>middle-man</em> tells me to talk to
stackoverflow folks. I go to them. They have answers, lots of them. Unfortunately (or not) sometimes none of the answers available help me solve the problem.
That is when I follow RTFM command.</p>
<!-- more -->
<p>I’m working on a Django project and got stuck when tried to get url reverse to work on templates.
All searches on google pointed to <a href="https://docs.djangoproject.com/en/dev/topics/http/urls/">this</a> page. I <strong><em>read</em></strong> it! At least I thought so.
After 2 evenings spent googling and changing code I decided to look into the source of django. I got illuminated. It looks like when I use url namespaces
I have to put a colon between the app and the view like <strong>app:index</strong>.</p>
<p>I got so frustrated and angry that I went to the page where I had spent so many time <em>reading</em> and searched for a colon (:). That was the time where I
as put on my place again, a very humble little place. I couldn’t believed I had missed <a href="https://docs.djangoproject.com/en/dev/topics/http/urls/#url-namespaces">this section</a>.</p>
<p>This is a warning to <strong>Read the Whole Fucking Manual</strong> because sometimes you miss important things that can easily spare you time and frustration.</p>
<p><a href="http://hugocaracol.github.io/code/rtfm-extended/">RTFM extended</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on June 08, 2013.</p>http://hugocaracol.github.io/oop/c/classes-in-c-programming2013-05-29T19:42:00+01:002013-05-29T19:42:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Last evening I was playing around with C and came up with an example showing simple OOP concepts.</p>
<p><strong>The working behaviour of a class is achieved with a <em>struct</em></strong>.
In this example I define a <em>struct</em> (or a class in OOP) to work with the abstraction data type <em>Person</em>.</p>
<!-- more -->
<div class="highlight"><pre><code class="c"><span class="k">struct</span> <span class="n">_Person</span><span class="p">{</span>
<span class="kt">char</span> <span class="o">*</span> <span class="n">name</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">age</span><span class="p">;</span>
<span class="c1">// methods</span>
<span class="kt">int</span> <span class="p">(</span><span class="o">*</span><span class="n">getAge</span><span class="p">)(</span><span class="n">Person</span><span class="p">);</span>
<span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">setAge</span><span class="p">)(</span><span class="n">Person</span><span class="p">,</span> <span class="kt">int</span><span class="p">);</span>
<span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">setName</span><span class="p">)(</span><span class="n">Person</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span><span class="p">);</span>
<span class="kt">char</span><span class="o">*</span> <span class="p">(</span><span class="o">*</span><span class="n">getName</span><span class="p">)(</span><span class="n">Person</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div>
<p>The first two fields are used to store some data (<em>name</em> and <em>age</em>) of a person.
The other fields are <strong>pointers to functions</strong>. This declaration of pointers to functions give this whole example an OOP style of coding.</p>
<p>When we create a new object of type <em>struct _Person</em>, we store it in the <strong>heap</strong>. Thus, we will refer to it as the pointer <em>Person</em>. </p>
<div class="highlight"><pre><code class="c"><span class="k">typedef</span> <span class="k">struct</span> <span class="n">_Person</span><span class="o">*</span> <span class="n">Person</span><span class="p">;</span>
</code></pre></div>
<p>As <em>struct _Person</em> contains pointers to functions, which have to exist first and then be assigned to the respective pointer in <em>struct _Person</em>.<br />
Below is one of the functions. <em>getAge</em> returns the age of <em>this</em> object (type Person).</p>
<div class="highlight"><pre><code class="c"><span class="kt">int</span> <span class="nf">getAge</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">this</span><span class="o">-></span><span class="n">age</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>To create an object of type Person, first we have to make some initializations and request some space in heap. We do all this, running:</p>
<div class="highlight"><pre><code class="c"><span class="n">Person</span> <span class="n">p</span> <span class="o">=</span> <span class="n">PersonNew</span><span class="p">();</span>
</code></pre></div>
<p>As we can see from the code below, <em>PersonNew</em> allocates memory and initializes the function pointers so they can be called as methods of a class.
In the end, the function returns a <strong>pointer to the new object</strong>.</p>
<div class="highlight"><pre><code class="c"><span class="n">Person</span> <span class="nf">PersonNew</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Person</span> <span class="n">this</span><span class="p">;</span>
<span class="n">this</span> <span class="o">=</span> <span class="n">malloc</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="k">struct</span> <span class="n">_Person</span><span class="p">));</span>
<span class="n">this</span><span class="o">-></span><span class="n">getAge</span> <span class="o">=</span> <span class="n">getAge</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">setAge</span> <span class="o">=</span> <span class="n">setAge</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">setName</span> <span class="o">=</span> <span class="n">setName</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">getName</span> <span class="o">=</span> <span class="n">getName</span><span class="p">;</span>
<span class="k">return</span> <span class="n">this</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>Now we have everything, to do the very basic of OOP in C.</p>
<p>Full working example below. <strong>Kudos to <a href="http://valgrind.org/">valgrind</a></strong> that helped me debug the code.
Beware that the heap space if not automatically freed. <strong>Enjoy :-)</strong></p>
<div class="highlight"><pre><code class="c"><span class="cp">#include <stdio.h></span>
<span class="cp">#include <stdlib.h></span>
<span class="cp">#include <string.h></span>
<span class="k">typedef</span> <span class="k">struct</span> <span class="n">_Person</span><span class="o">*</span> <span class="n">Person</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">_Person</span><span class="p">{</span>
<span class="kt">char</span> <span class="o">*</span> <span class="n">name</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">age</span><span class="p">;</span>
<span class="c1">// methods</span>
<span class="kt">int</span> <span class="p">(</span><span class="o">*</span><span class="n">getAge</span><span class="p">)(</span><span class="n">Person</span><span class="p">);</span>
<span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">setAge</span><span class="p">)(</span><span class="n">Person</span><span class="p">,</span> <span class="kt">int</span><span class="p">);</span>
<span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">setName</span><span class="p">)(</span><span class="n">Person</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span><span class="p">);</span>
<span class="kt">char</span><span class="o">*</span> <span class="p">(</span><span class="o">*</span><span class="n">getName</span><span class="p">)(</span><span class="n">Person</span><span class="p">);</span>
<span class="p">};</span>
<span class="kt">int</span> <span class="nf">getAge</span><span class="p">(</span><span class="n">Person</span> <span class="n">p</span><span class="p">);</span>
<span class="kt">void</span> <span class="nf">setAge</span><span class="p">(</span><span class="n">Person</span> <span class="n">p</span><span class="p">,</span> <span class="kt">int</span> <span class="n">age</span><span class="p">);</span>
<span class="kt">char</span><span class="o">*</span> <span class="nf">getName</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">);</span>
<span class="kt">void</span> <span class="nf">setName</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span> <span class="n">name</span><span class="p">);</span>
<span class="n">Person</span> <span class="nf">PersonNew</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Person</span> <span class="n">this</span><span class="p">;</span>
<span class="n">this</span> <span class="o">=</span> <span class="n">malloc</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="k">struct</span> <span class="n">_Person</span><span class="p">));</span>
<span class="n">this</span><span class="o">-></span><span class="n">getAge</span> <span class="o">=</span> <span class="n">getAge</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">setAge</span> <span class="o">=</span> <span class="n">setAge</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">setName</span> <span class="o">=</span> <span class="n">setName</span><span class="p">;</span>
<span class="n">this</span><span class="o">-></span><span class="n">getName</span> <span class="o">=</span> <span class="n">getName</span><span class="p">;</span>
<span class="k">return</span> <span class="n">this</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="nf">PersonDestroy</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">free</span><span class="p">(</span><span class="n">this</span><span class="o">-></span><span class="n">name</span><span class="p">);</span>
<span class="n">free</span><span class="p">(</span><span class="n">this</span><span class="p">);</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="nf">getAge</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">this</span><span class="o">-></span><span class="n">age</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="nf">setAge</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">,</span> <span class="kt">int</span> <span class="n">age</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">this</span><span class="o">-></span><span class="n">age</span> <span class="o">=</span> <span class="n">age</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">char</span><span class="o">*</span> <span class="nf">getName</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">this</span><span class="o">-></span><span class="n">name</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="nf">setName</span><span class="p">(</span><span class="n">Person</span> <span class="n">this</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span> <span class="n">name</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">this</span><span class="o">-></span><span class="n">name</span> <span class="o">=</span> <span class="n">malloc</span><span class="p">(</span><span class="n">strlen</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="n">strcpy</span><span class="p">(</span><span class="n">this</span><span class="o">-></span><span class="n">name</span><span class="p">,</span> <span class="n">name</span><span class="p">);</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Person</span> <span class="n">p</span> <span class="o">=</span> <span class="n">PersonNew</span><span class="p">();</span>
<span class="n">p</span><span class="o">-></span><span class="n">setAge</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="mi">70</span><span class="p">);</span>
<span class="n">p</span><span class="o">-></span><span class="n">setName</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="s">"Hugo Santos"</span><span class="p">);</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"%d</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">p</span><span class="o">-></span><span class="n">getAge</span><span class="p">(</span><span class="n">p</span><span class="p">));</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"%s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">p</span><span class="o">-></span><span class="n">getName</span><span class="p">(</span><span class="n">p</span><span class="p">));</span>
<span class="n">PersonDestroy</span><span class="p">(</span><span class="n">p</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p><a href="http://hugocaracol.github.io/oop/c/classes-in-c-programming/">Classes in C Programming</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on May 29, 2013.</p>http://hugocaracol.github.io/code/fibonacci-ultra-fast2013-05-21T23:27:00+01:002013-05-21T23:27:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>It’s never too late to optimize the code. Never too late to realize there’s a better approach.</p>
<p>After writing the last post I came across a different approach. If we compute the fibonacci number (n - 1),
that computing also computes the fib number (n - 2). Therefore we don’t need to compute ir again. </p>
<!-- more -->
<p>Look at the code. Now we use 2 functions.
One function that is called only once and another one that’s called iteratively.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">fib_fast_ultra</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">global</span> <span class="n">fib_dict</span>
<span class="c"># clears global dictionary</span>
<span class="n">fib_dict</span><span class="o">.</span><span class="n">clear</span><span class="p">()</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">_fib_fast_ultra_iter</span><span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">fib_dict</span><span class="p">[</span><span class="n">n</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span>
<span class="k">return</span> <span class="n">n_1</span> <span class="o">+</span> <span class="n">n_2</span>
<span class="k">def</span> <span class="nf">_fib_fast_ultra_iter</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">global</span> <span class="n">fib_dict</span>
<span class="k">if</span> <span class="n">n</span> <span class="o"><</span> <span class="mi">2</span><span class="p">:</span>
<span class="k">return</span> <span class="n">n</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="ow">in</span> <span class="n">fib_dict</span><span class="p">:</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">fib_dict</span><span class="p">[</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">_fib_fast_ultra_iter</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">fib_dict</span><span class="p">[(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)]</span> <span class="o">=</span> <span class="n">n_1</span>
<span class="k">if</span> <span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="ow">in</span> <span class="n">fib_dict</span><span class="p">:</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">fib_dict</span><span class="p">[</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">_fib_fast_ultra_iter</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">fib_dict</span><span class="p">[(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)]</span> <span class="o">=</span> <span class="n">n_2</span>
<span class="k">return</span> <span class="n">n_1</span> <span class="o">+</span> <span class="n">n_2</span>
</code></pre></div>
<p>Running the function above and the other shown <a href="http://hugocaracol.github.io/blog/2013/05/21/accelerating-with-memoization">here</a> we see the magic!
Computing the fibonacci number 500 using the function from the previous post
we get 0.0014700890 seconds. But if we run it with this function we get 0.0011551380 seconds. That is an <strong>increasing performance of 78.6% improvement.</strong></p>
<p><a href="http://hugocaracol.github.io/code/fibonacci-ultra-fast/">Fibonacci ultra fast</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on May 21, 2013.</p>http://hugocaracol.github.io/code/accelerating-with-memoization2013-05-21T20:20:00+01:002013-05-21T20:20:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Memoization can be incredible useful for expensive calculations. Just look at the results at the bottom of the post to check how long the 2 functions (with and without memoization) take to run the computation of fib 35.</p>
<!-- more -->
<div class="highlight"><pre><code class="python"><span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">fib_slow</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">if</span> <span class="n">n</span> <span class="o"><</span> <span class="mi">2</span><span class="p">:</span>
<span class="k">return</span> <span class="n">n</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">fib_slow</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">fib_slow</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span>
<span class="k">return</span> <span class="n">n_1</span> <span class="o">+</span> <span class="n">n_2</span>
<span class="n">fib_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">def</span> <span class="nf">fib_fast</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">global</span> <span class="n">fib_dict</span>
<span class="k">if</span> <span class="n">n</span> <span class="o"><</span> <span class="mi">2</span><span class="p">:</span>
<span class="k">return</span> <span class="n">n</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="p">(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="ow">in</span> <span class="n">fib_dict</span><span class="p">:</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">fib_dict</span><span class="p">[</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">n_1</span> <span class="o">=</span> <span class="n">fib_fast</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">fib_dict</span><span class="p">[(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)]</span> <span class="o">=</span> <span class="n">n_1</span>
<span class="k">if</span> <span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="ow">in</span> <span class="n">fib_dict</span><span class="p">:</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">fib_dict</span><span class="p">[</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">n_2</span> <span class="o">=</span> <span class="n">fib_fast</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">fib_dict</span><span class="p">[(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)]</span> <span class="o">=</span> <span class="n">n_2</span>
<span class="k">return</span> <span class="n">n_1</span> <span class="o">+</span> <span class="n">n_2</span>
</code></pre></div>
<p>Running…</p>
<div class="highlight"><pre><code class="python"><span class="n">n</span> <span class="o">=</span> <span class="mi">35</span>
<span class="k">print</span> <span class="s">"Calculating fib of "</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="k">print</span> <span class="s">"Using fib_slow (without memoization)"</span>
<span class="n">start_time</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="k">print</span> <span class="n">fib_slow</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="k">print</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">start_time</span><span class="p">,</span> <span class="s">"seconds"</span>
<span class="k">print</span> <span class="s">"Accelerating... (with memoization)"</span>
<span class="n">start_time</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="k">print</span> <span class="n">fib_fast</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'{0:.10f}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">start_time</span><span class="p">),</span> <span class="s">"seconds"</span>
</code></pre></div>
<p>we get…</p>
<p>Calculating fib of 35<br />
Using fib_slow (without memoization)<br />
9227465<br />
11.9559807777 seconds<br />
Accelerating… (with memoization)<br />
9227465<br />
<strong>0.0000698566 seconds</strong></p>
<p><a href="http://hugocaracol.github.io/code/accelerating-with-memoization/">Accelerating with memoization</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on May 21, 2013.</p>http://hugocaracol.github.io/scheme/lisp/sqrt-newton-way2013-05-12T00:24:00+01:002013-05-12T00:24:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>I was working through a way to get a square-root function to work in scheme.</p>
<div class="highlight"><pre><code class="scheme"><span class="p">(</span><span class="k">define </span><span class="p">(</span><span class="nf">sqrt2</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="nf">sqrt-iter</span> <span class="mi">1</span> <span class="nv">x</span><span class="p">))</span>
<span class="p">(</span><span class="k">define </span><span class="p">(</span><span class="nf">sqrt-iter</span> <span class="nv">guess</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nf">good-enough</span> <span class="nv">guess</span> <span class="nv">x</span><span class="p">)</span>
<span class="nv">guess</span>
<span class="p">(</span><span class="nf">sqrt-iter</span> <span class="p">(</span><span class="nf">make-guess</span> <span class="nv">guess</span> <span class="nv">x</span><span class="p">)</span> <span class="nv">x</span><span class="p">)))</span>
<span class="p">(</span><span class="k">define </span><span class="p">(</span><span class="nf">make-guess</span> <span class="nv">guess</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="nb">/ </span><span class="p">(</span><span class="nb">+ </span><span class="p">(</span><span class="nb">/ </span><span class="nv">x</span> <span class="nv">guess</span><span class="p">)</span> <span class="nv">guess</span><span class="p">)</span> <span class="mi">2</span><span class="p">))</span>
<span class="p">(</span><span class="k">define </span><span class="p">(</span><span class="nf">good-enough</span> <span class="nv">guess</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="nb">< </span><span class="p">(</span><span class="nf">abs2</span> <span class="p">(</span><span class="nf">-</span><span class="p">(</span><span class="nb">* </span><span class="nv">guess</span> <span class="nv">guess</span><span class="p">)</span> <span class="nv">x</span><span class="p">))</span> <span class="mf">0.001</span><span class="p">))</span>
<span class="p">(</span><span class="k">define </span><span class="p">(</span><span class="nf">abs2</span> <span class="nv">x</span><span class="p">)</span>
<span class="p">(</span><span class="k">if </span><span class="p">(</span><span class="nb">< </span><span class="nv">x</span> <span class="mi">0</span><span class="p">)</span>
<span class="p">(</span><span class="nb">- </span><span class="mi">0</span> <span class="nv">x</span><span class="p">)</span>
<span class="nv">x</span><span class="p">))</span>
</code></pre></div>
<p>Running: </p>
<div class="highlight"><pre><code class="scheme"><span class="p">(</span><span class="nf">sqrt2</span> <span class="mi">100</span><span class="p">)</span>
</code></pre></div>
<p>I got the number 10 followed by the precision: 139008452377144732764939786789661303114218850808529137991604824430036072629766435941001769154109609521811665540548899435521 / 993650612510151824074958925538589300439185013461061345483147770187307778405188218146967823138235109552370030745199319835640898137728</p>
<p>A precision I won’t forget. COOL!!!</p>
<p><a href="http://hugocaracol.github.io/scheme/lisp/sqrt-newton-way/">sqrt newton way</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on May 12, 2013.</p>http://hugocaracol.github.io/debugging/delta-debugging2013-04-09T23:31:00+01:002013-04-09T23:31:00+01:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Delta Debugging is an automated debugging approach based on systematic testing.</p>
<p>Programmers follow this approach when they are manually debugging a piece of code that is failing at some point. Imagine some large input is causing a function to crash. What do you do? You start by reducing the input and check if the function works. If not, you do the chopping again until it works.<br />
This can be very hard to do if some random set from a 5000 character string is failing. It’s like the needle in a haystack problem.</p>
<!-- more -->
<p>Well, delta debugging simplifies this A LOT! The algorithm <strong>starts by chopping the input in 2</strong> (the granularity - n). Then, it removes the first substring and tests with the second one. If the test passes, it runs now with only the first substring. The latter caused the test to fail. Good. Now we know that the problem is in there. The algorithm goes on using that substring (the one that failed the test) and, again, chopps it in two. But on the next step, both parts of the <em>failed substring</em> pass the test. When this occurrs the granularity value (n) is <strong>updated to n * 2</strong> and the algorithm continues until it finds the minimal input that is causing the failure. This is a very simple overview of how delta debugging works.</p>
<p>One importart part of this equation is the test function. We need a good function in order to get good results. You can create a generic test function that executes some code and traps exceptions. This way the test function can return true or false or even PASS or FAIL. Wichever suits you.</p>
<p>Below I give you an example of this code (provided by Professor Andreas Zeller). The test function checks wether the html select tag is found in the input string. It is obvious in this case but in practice you don’t know what is causing the problem.</p>
<div class="highlight"><pre><code class="python"><span class="kn">import</span> <span class="nn">re</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="k">global</span> <span class="n">counter</span>
<span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="s">"<SELECT[^>]*>"</span><span class="p">,</span> <span class="n">s</span><span class="p">)</span> <span class="o">>=</span> <span class="mi">0</span><span class="p">:</span>
<span class="k">print</span> <span class="s">"test "</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span> <span class="o">+</span> <span class="s">": "</span><span class="p">,</span> <span class="n">s</span><span class="p">,</span> <span class="s">"FAIL"</span>
<span class="k">return</span> <span class="s">"FAIL"</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">print</span> <span class="s">"test "</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span> <span class="o">+</span> <span class="s">": "</span><span class="p">,</span> <span class="n">s</span><span class="p">,</span> <span class="s">"PASS"</span>
<span class="k">return</span> <span class="s">"PASS"</span>
<span class="k">def</span> <span class="nf">ddmin</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">test</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="o">==</span> <span class="s">"FAIL"</span>
<span class="n">n</span> <span class="o">=</span> <span class="mi">2</span> <span class="c"># Initial granularity</span>
<span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
<span class="k">print</span> <span class="s">"Using n="</span><span class="p">,</span> <span class="n">n</span>
<span class="n">start</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">subset_length</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="o">/</span> <span class="n">n</span>
<span class="k">print</span> <span class="s">"Using subset_length="</span><span class="p">,</span> <span class="n">subset_length</span>
<span class="n">some_complement_is_failing</span> <span class="o">=</span> <span class="bp">False</span>
<span class="k">while</span> <span class="n">start</span> <span class="o"><</span> <span class="nb">len</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="n">complement</span> <span class="o">=</span> <span class="n">s</span><span class="p">[:</span><span class="n">start</span><span class="p">]</span> <span class="o">+</span> <span class="n">s</span><span class="p">[</span><span class="n">start</span> <span class="o">+</span> <span class="n">subset_length</span><span class="p">:]</span>
<span class="k">if</span> <span class="n">test</span><span class="p">(</span><span class="n">complement</span><span class="p">)</span> <span class="o">==</span> <span class="s">"FAIL"</span><span class="p">:</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">complement</span>
<span class="n">n</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">some_complement_is_failing</span> <span class="o">=</span> <span class="bp">True</span>
<span class="k">break</span>
<span class="n">start</span> <span class="o">+=</span> <span class="n">subset_length</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">some_complement_is_failing</span><span class="p">:</span>
<span class="k">if</span> <span class="n">n</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>
<span class="k">break</span>
<span class="n">n</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">n</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">s</span><span class="p">))</span>
<span class="k">return</span> <span class="n">s</span>
</code></pre></div>
<div class="highlight"><pre><code class="python"><span class="n">html_input</span> <span class="o">=</span> <span class="s">'<SELECT>foo</SELECT>'</span>
<span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">print</span> <span class="n">ddmin</span><span class="p">(</span><span class="n">html_input</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'called test '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span> <span class="o">+</span> <span class="s">' times'</span>
</code></pre></div>
<div class="highlight"><pre><code class="python"><span class="n">test</span> <span class="mi">1</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECT</span><span class="o">></span><span class="n">foo</span><span class="o"></</span><span class="n">SELECT</span><span class="o">></span> <span class="n">FAIL</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">2</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">10</span>
<span class="n">test</span> <span class="mi">2</span><span class="p">:</span> <span class="n">o</span><span class="o"></</span><span class="n">SELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">3</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECT</span><span class="o">></span><span class="n">fo</span> <span class="n">FAIL</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">2</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">5</span>
<span class="n">test</span> <span class="mi">4</span><span class="p">:</span> <span class="n">CT</span><span class="o">></span><span class="n">fo</span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">5</span><span class="p">:</span> <span class="o"><</span><span class="n">SELE</span> <span class="n">PASS</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">4</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">2</span>
<span class="n">test</span> <span class="mi">6</span><span class="p">:</span> <span class="n">ELECT</span><span class="o">></span><span class="n">fo</span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">7</span><span class="p">:</span> <span class="o"><</span><span class="n">SECT</span><span class="o">></span><span class="n">fo</span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">8</span><span class="p">:</span> <span class="o"><</span><span class="n">SELT</span><span class="o">></span><span class="n">fo</span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">9</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECfo</span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">10</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECT</span><span class="o">></span> <span class="n">FAIL</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">3</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">2</span>
<span class="n">test</span> <span class="mi">11</span><span class="p">:</span> <span class="n">ELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">12</span><span class="p">:</span> <span class="o"><</span><span class="n">SECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">13</span><span class="p">:</span> <span class="o"><</span><span class="n">SELT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">14</span><span class="p">:</span> <span class="o"><</span><span class="n">SELEC</span> <span class="n">PASS</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">6</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">1</span>
<span class="n">test</span> <span class="mi">15</span><span class="p">:</span> <span class="n">SELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">16</span><span class="p">:</span> <span class="o"><</span><span class="n">ELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">17</span><span class="p">:</span> <span class="o"><</span><span class="n">SLECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">18</span><span class="p">:</span> <span class="o"><</span><span class="n">SEECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">19</span><span class="p">:</span> <span class="o"><</span><span class="n">SELCT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">20</span><span class="p">:</span> <span class="o"><</span><span class="n">SELET</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">21</span><span class="p">:</span> <span class="o"><</span><span class="n">SELEC</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">22</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECT</span> <span class="n">PASS</span>
<span class="n">Using</span> <span class="n">n</span><span class="o">=</span> <span class="mi">8</span>
<span class="n">Using</span> <span class="n">subset_length</span><span class="o">=</span> <span class="mi">1</span>
<span class="n">test</span> <span class="mi">23</span><span class="p">:</span> <span class="n">SELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">24</span><span class="p">:</span> <span class="o"><</span><span class="n">ELECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">25</span><span class="p">:</span> <span class="o"><</span><span class="n">SLECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">26</span><span class="p">:</span> <span class="o"><</span><span class="n">SEECT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">27</span><span class="p">:</span> <span class="o"><</span><span class="n">SELCT</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">28</span><span class="p">:</span> <span class="o"><</span><span class="n">SELET</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">29</span><span class="p">:</span> <span class="o"><</span><span class="n">SELEC</span><span class="o">></span> <span class="n">PASS</span>
<span class="n">test</span> <span class="mi">30</span><span class="p">:</span> <span class="o"><</span><span class="n">SELECT</span> <span class="n">PASS</span>
<span class="o"><</span><span class="n">SELECT</span><span class="o">></span>
<span class="n">called</span> <span class="n">test</span> <span class="mi">30</span> <span class="n">times</span>
</code></pre></div>
<p>For the people using <strong>git</strong> there’s <strong>git bisect</strong>. Some people say that:<br />
>”git bisect” is one of the most useful and overlooked debugging tools.</p>
<p><a href="http://hugocaracol.github.io/debugging/delta-debugging/">Delta Debugging</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on April 09, 2013.</p>http://hugocaracol.github.io/bitwise/bitwise-and-constant-values2013-02-15T22:29:00+00:002013-02-15T22:29:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Recently I read an article explaining how to use bitwise operations to grant or deny access to something.
I liked the article so much that I’ll try to add it to my personal stack (my blog).</p>
<p>There are 4 types of bitwise operators: <strong>NOT</strong>, <strong>AND</strong>, <strong>OR</strong> and <strong>XOR</strong>.
In Python these operators are <strong>~</strong>, <strong>&</strong>, <strong>|</strong>, <strong>^</strong> respectively.</p>
<h2 id="truth-tables">Truth tables</h2>
<h4 id="nbspnbspnotnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspandnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspornbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspnbspxor"> NOT AND OR XOR</h4>
<table>
<tr>
<td>
<table style="border: 1px solid black;">
<tr>
<th style="border: 1px solid black;">A</th><th style="border: 1px solid black;text-align:center">NOT A</th>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td><td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td><td style="border: 1px solid black;text-align:center">1</td>
</tr>
</table>
</td>
<td> </td>
<td>
<table style="border: 1px solid black;">
<tr>
<th style="border: 1px solid black;text-align:center">A</th>
<th style="border: 1px solid black;text-align:center">B</th>
<th style="border: 1px solid black;text-align:center">A AND B</th>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
</table>
</td>
<td> </td>
<td>
<table style="border: 1px solid black;">
<tr>
<th style="border: 1px solid black;text-align:center">A</th>
<th style="border: 1px solid black;text-align:center">B</th>
<th style="border: 1px solid black;text-align:center">A OR B</th>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
</table>
</td>
<td> </td>
<td>
<table style="border: 1px solid black;">
<tr>
<th style="border: 1px solid black;text-align:center">A</th>
<th style="border: 1px solid black;text-align:center">B</th>
<th style="border: 1px solid black;text-align:center">A XOR B</th>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">0</td>
<td style="border: 1px solid black;text-align:center">1</td>
</tr>
<tr>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">1</td>
<td style="border: 1px solid black;text-align:center">0</td>
</tr>
</table>
</td>
</tr>
</table>
<h2 id="left-shift-right-shift">Left Shift, Right Shift</h2>
<p>If we have a byte representing an unsigned integer 1 (0 0 0 0 0 0 0 1) and make a left shift, we get 2 (0 0 0 0 0 0 1 0). If we make another shift to the left
we get 4 (0 0 0 0 0 1 0 0) and so on.</p>
<p>On the other hand if we have a byte with the unsigned integer 4 (0 0 0 0 0 1 0 0) and shift it to the right, we get 2 (0 0 0 0 0 0 1 0).</p>
<p>In Python <strong>«</strong> represents left shift and <strong>»</strong> represents right shift.
<!-- more --></p>
<h2 id="assigning-constant-values">Assigning CONSTANT values</h2>
<p>Imagine we need to set some permissions to protect some sort of CRUD (Create Read Update Delete) system. Different users have different permissions.
One user may have more than one permission. If he’s root (or superman) he may do all 4, but if he’s a guest he may only read.</p>
<p>To set all this stuff easily we can start by assigning PERM_CREATE = 1. Next is PERM_READ. But which value this constant will hold? Easy!!
We can left shift PERM_CREATE and assigning it to PERM_READ. Therefore, PERM_READ = 2. And we do the same thing for the remaining permission items.</p>
<p>In the end we get:<br />
PERM_CREATE = 1 (<strong>0 0 0 0 0 0 0 1</strong>)<br />
PERM_READ = 2 (<strong>0 0 0 0 0 0 1 0</strong>)<br />
PERM_UPDATE = 4 (<strong>0 0 0 0 0 1 0 0</strong>)<br />
PERM_DELETE = 8 (<strong>0 0 0 0 1 0 0 0</strong>) </p>
<p>Did you follow the binary pattern represented above? Noticed all the left shifts?</p>
<h2 id="making-sense-of-these-constants">Making sense of these constants</h2>
<p>To give all permissions to the user A we can do:</p>
<div class="highlight"><pre><code class="python"><span class="n">PERM_A</span> <span class="o">=</span> <span class="n">PERM_CREATE</span> <span class="o">|</span> <span class="n">PERM_READ</span> <span class="o">|</span> <span class="n">PERM_UPDATE</span> <span class="o">|</span> <span class="n">PERM_DELETE</span>
</code></pre></div>
<p>If we check the value of PERM_A, we get 15.</p>
<p>0 0 0 0 0 0 0 1 <strong>PERM_CREATE</strong><br />
0 0 0 0 0 0 1 0 <strong>PERM_READ</strong><br />
0 0 0 0 0 1 0 0 <strong>PERM_UPDATE</strong><br />
0 0 0 0 1 0 0 0 <strong>PERM_DELETE</strong><br />
=-=-=-=-=-=-=-=<br />
0 0 0 0 1 1 1 1 => 15 (base 10) </p>
<p>Having PERM_A all permissions, if we want to remove the permission to delete we just ‘<strong>XOR</strong> it’. </p>
<div class="highlight"><pre><code class="python"><span class="n">PERM_A</span> <span class="o">=</span> <span class="n">PERM_A</span> <span class="o">^</span> <span class="n">PERM_DELETE</span>
</code></pre></div>
<p>And finally, to check if the user has permission to delete we just ‘<strong>AND</strong> it’.</p>
<div class="highlight"><pre><code class="python"><span class="k">if</span> <span class="n">PERM_A</span> <span class="o">&</span> <span class="n">PERM_DELETE</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'Yes I can!'</span>
</code></pre></div>
<p>This concludes the bitwise perm system.</p>
<p><a href="http://hugocaracol.github.io/bitwise/bitwise-and-constant-values/">Bitwise and CONSTANT values</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 15, 2013.</p>http://hugocaracol.github.io/debugging/making-of-a-simple-debugger2013-02-12T23:50:00+00:002013-02-12T23:50:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>I was introduced to sys.settrace in the Udacity course on debugging. Python docs say:
>Set the system’s trace function, which allows you to implement a Python source code debugger in Python. </p>
<p>With this function, is very easy to make a simple debugger. Even if not implemented to use in a production environment,
it’s still a funny thing to do.</p>
<p>To make use of this, a trace function must be created. A trace function has 3 arguments:
<em>frame</em>, <em>event</em> and <em>arg</em>. <em>frame</em> represents the current stack. <em>event</em> is a string like
‘call’, ‘line’, ‘return’, ‘exception’ and some others. <em>arg</em> is used when event is ‘return’. </p>
<!-- more -->
<h3 id="events">Events</h3>
<ul>
<li>‘call’ - triggered when a function is called</li>
<li>‘line’ - triggered when the interpreter is about to execute a new line of code</li>
<li>‘return’ - triggered when a function is about to return. The return value is stored in <em>arg</em>.</li>
<li>‘exception’ - triggered when an exception occurrs. A tuple like (exception, value, traceback) is stored in <em>arg</em>.</li>
</ul>
<p>In the example that follows I use the first 3 events.</p>
<h3 id="debugger">Debugger</h3>
<div class="highlight"><pre><code class="python"><span class="k">class</span> <span class="nc">Debugger</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">prev_vars</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">def</span> <span class="nf">_get_changed_vars</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">f_locals</span><span class="p">):</span>
<span class="n">changed_vars</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">f_locals</span><span class="o">.</span><span class="n">iteritems</span><span class="p">():</span>
<span class="k">if</span> <span class="n">k</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">prev_vars</span><span class="p">:</span>
<span class="k">if</span> <span class="n">v</span> <span class="o">!=</span> <span class="bp">self</span><span class="o">.</span><span class="n">prev_vars</span><span class="p">[</span><span class="n">k</span><span class="p">]:</span>
<span class="n">changed_vars</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">v</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">changed_vars</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">v</span>
<span class="c"># updates self.vars to the dictionary of current variable</span>
<span class="bp">self</span><span class="o">.</span><span class="n">prev_vars</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">v</span>
<span class="k">return</span> <span class="n">changed_vars</span>
<span class="k">def</span> <span class="nf">track</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">frame</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">arg</span><span class="p">):</span>
<span class="n">prefix</span> <span class="o">=</span> <span class="s">"[DEBUG] "</span>
<span class="k">if</span> <span class="n">event</span> <span class="o">==</span> <span class="s">'line'</span><span class="p">:</span>
<span class="k">print</span> <span class="n">prefix</span> <span class="o">+</span> <span class="s">"Line no: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">frame</span><span class="o">.</span><span class="n">f_lineno</span>
<span class="n">changed_vars</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_changed_vars</span><span class="p">(</span><span class="n">frame</span><span class="o">.</span><span class="n">f_locals</span><span class="p">)</span>
<span class="k">if</span> <span class="n">changed_vars</span><span class="p">:</span>
<span class="k">print</span> <span class="n">prefix</span> <span class="o">+</span> <span class="s">"Changed variables: </span><span class="si">%s</span><span class="s"> "</span> <span class="o">%</span> <span class="n">changed_vars</span>
<span class="k">elif</span> <span class="n">event</span> <span class="o">==</span> <span class="s">'call'</span><span class="p">:</span>
<span class="k">print</span> <span class="n">prefix</span> <span class="o">+</span> <span class="s">"Called: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">frame</span><span class="o">.</span><span class="n">f_code</span><span class="o">.</span><span class="n">co_name</span>
<span class="k">elif</span> <span class="n">event</span> <span class="o">==</span> <span class="s">'return'</span><span class="p">:</span>
<span class="k">print</span> <span class="n">prefix</span> <span class="o">+</span> <span class="s">"Exiting: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">frame</span><span class="o">.</span><span class="n">f_code</span><span class="o">.</span><span class="n">co_name</span>
<span class="k">print</span> <span class="n">prefix</span> <span class="o">+</span> <span class="s">" Returning: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">arg</span>
<span class="c"># for every event, the trace function is returned</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">track</span>
</code></pre></div>
<p>I’ve implemented a class because I wanted to track the value change of variables (just the function <em>track</em> alone should be enough to get the gist of it).<br />
The <strong>track</strong> function does all the magic. As you can see, it has the 3 arguments pointed before.<br />
When some event occurrs this function is triggered. We watch for the event and act accordingly, printing variables, lines or return values.
<strong>This function must return itself</strong> in order to keep the trace going.
I have not implemented breakpoints but with the line numbers it’s a very easy thing to do.
<strong>_get_changed_vars</strong> function is just a helper function I use to get the new or updated variables.</p>
<p>Now, to use our shiny debugger we create a new instance of our class and pass <em>debugger.track</em> as a parameter to <em>settrace</em>. Then we invoke the code
we want to debug and in the end we disable <em>settrace</em>.</p>
<div class="highlight"><pre><code class="python"><span class="n">debugger</span> <span class="o">=</span> <span class="n">Debugger</span><span class="p">()</span>
<span class="n">sys</span><span class="o">.</span><span class="n">settrace</span><span class="p">(</span><span class="n">debugger</span><span class="o">.</span><span class="n">track</span><span class="p">)</span>
<span class="c"># call your function here</span>
<span class="n">sys</span><span class="o">.</span><span class="n">settrace</span><span class="p">(</span><span class="bp">None</span><span class="p">)</span>
</code></pre></div>
<p>This debugger can be extended to infer invariants (one of the exercises on the debugging course). Watching how variables change, can give us hints about
their ranges (min, max). Use it but don’t take it too seriously ;-)</p>
<p><a href="http://hugocaracol.github.io/debugging/making-of-a-simple-debugger/">Making of a simple debugger</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 12, 2013.</p>http://hugocaracol.github.io/debugging/preconditions-postconditions-invariants2013-02-11T18:56:00+00:002013-02-11T18:56:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Frequently when debugging software we usually spend lots of time figuring out where exactly is the bug that is creating the
failure of the software. Maybe it’s where the failure occurs but probably it’s not. Bugs can infect parts of software and
these parts can cause failure of other parts. Sometimes finding a bug is like finding a needle in a haystack.</p>
<p>These 3 topics belong to the notion of design-by-contract (DbC) programming.
Using this approach to design software, programmers should define code contracts - conditions
that hold true before a method is run (<strong>preconditions</strong>), or after the method is run (<strong>postconditions</strong>) or both (<strong>invariants</strong>).
Eiffel programming language was built around this concept (among others).</p>
<!-- more -->
<h3 id="preconditions">Preconditions</h3>
<p>Set of conditions that must assert to True before running the code in a method. These conditions can be, for example, variables that must
have a positive value in order to the code in the method behave correctly.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">square_root</span><span class="p">(</span><span class="n">number</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">number</span> <span class="o">>=</span> <span class="mi">0</span>
<span class="o">...</span>
</code></pre></div>
<h3 id="postconditions">Postconditions</h3>
<p>Set of conditions that must assert to True after the code in a method is run. These conditions verify the correct execution of the method code.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">square_root</span><span class="p">(</span><span class="n">number</span><span class="p">):</span>
<span class="o">...</span>
<span class="c"># variable root holds the square root value of number</span>
<span class="k">assert</span> <span class="n">root</span> <span class="o">*</span> <span class="n">root</span> <span class="o">==</span> <span class="n">number</span> <span class="c"># i'm ignoring precision for readability purposes</span>
<span class="k">return</span> <span class="n">root</span>
</code></pre></div>
<h3 id="invariants">Invariants</h3>
<p>Set of conditions that must assert to True before and after the execution of a method. Invariants are composed by a set of conditions that are always valid. This is
useful to ensure that the code is running according to the algorithm.</p>
<p><a href="http://hugocaracol.github.io/debugging/preconditions-postconditions-invariants/">Preconditions, Postconditions and Invariants</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 11, 2013.</p>http://hugocaracol.github.io/debugging/software-debugging2013-02-08T23:22:00+00:002013-02-08T23:22:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>I started <a href="https://www.udacity.com/course/cs259">Software debugging class</a> on Udacity.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/8/8a/H96566k.jpg" alt="First bug being found" /></p>
<h3 id="syllabus">Syllabus</h3>
<!-- more -->
<h4 id="unit-1-how-debuggers-work">Unit 1: How Debuggers work</h4>
<p>Theory: Scientific method and its application to debugging.<br />
Fun fact: First bug in the history of computer science.<br />
Practice: Building a simple tracer.</p>
<h4 id="unit-2-asserting-expectations">Unit 2: Asserting Expectations</h4>
<p>Theory: Assertions in testing and in debugging.<br />
Fun fact: The most expensive bug in history.<br />
Practice: Improving the tracer.</p>
<h4 id="unit-3-simplifying-failures">Unit 3: Simplifying Failures</h4>
<p>Theory: Strategy of simplifying failures. Binary search. Delta debugging principle.<br />
Fun fact: Mozilla bugathon.<br />
Practice: Building a delta debugger.</p>
<h4 id="unit-4-tracking-origins">Unit 4: Tracking Origins</h4>
<p>Theory: Cause-effect chain. Deduction. Dependencies. Slices.<br />
Fun fact: Sherlock Holmes and Doctor Watson.<br />
Practice: Improving the delta debugger.</p>
<h4 id="unit-5-reproducing-failures">Unit 5: Reproducing Failures</h4>
<p>Theory: Types of bugs (Bohr bug, Heisenbug, Mandelbug, Schrodinbug). Systematic reproduction process.<br />
Fun fact: Mad laptop bug.<br />
Practice: Building a statistic debugging tool.</p>
<h4 id="unit-6-learning-from-mistakes">Unit 6: Learning from Mistakes</h4>
<p>Theory: Bug database management. Classifying bugs. Bug maps. Learning from mistakes.<br />
Fun fact: Programmer with the most buggy code.<br />
Practice: Improving your tools and practicing on a real world bug database.</p>
<p><a href="http://hugocaracol.github.io/debugging/software-debugging/">Software Debugging</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 08, 2013.</p>http://hugocaracol.github.io/testing/babysitting-an-army-of-monkeys2013-02-06T23:19:00+00:002013-02-06T23:19:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>“Babysitting an Army of Monkeys: An analysis of fuzzing 4 products with 5 lines of Python” was the title of a talk by Charlie Miller in CodenomiCON 2010.<br />
Mr Miller talked about how he broke some software products with only just a few lines of Python code.
The steps are:</p>
<ol>
<li>load a file into a buffer</li>
<li>at a random position of the buffer change the byte to a random one (5 lines of python code)</li>
<li>save the buffer</li>
<li>run the process</li>
<li>look at the exit code</li>
<li>if it doesn’t die (no bug found)</li>
<li>start over</li>
</ol>
<p>I embedded the youtube videos (Part1 and Part2). Click “Read On”!
<!-- more -->
<iframe width="560" height="315" src="http://www.youtube.com/embed/Xnwodi2CBws" frameborder="0" allowfullscreen></iframe></p>
<iframe width="560" height="315" src="http://www.youtube.com/embed/lK5fgCvS2N4" frameborder="0" allowfullscreen=""></iframe>
<p><a href="http://hugocaracol.github.io/testing/babysitting-an-army-of-monkeys/">Babysitting an army of monkeys</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 06, 2013.</p>http://hugocaracol.github.io/testing/code-coverage2013-02-04T22:42:00+00:002013-02-04T22:42:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Code coverage is a metric used in software testing that describes the percentage of code that’s been tested.
Some types of code coverage are: function coverage, statement coverage, branch coverage, loop coverage, among others.</p>
<h3 id="function-coverage">Function coverage</h3>
<p>It gives us a percentage of the functions being called by our tests. If we have 100 functions but only 50 are triggered by our test suite,
the function coverage is 50%. This score can be improved by modifying our tests so that every function is executed at least once. The function
coverage is also good to inform us if we have “dead functions” (functions that never get to be executed) in our code.</p>
<h3 id="statement-coverage">Statement coverage</h3>
<p>As the name says, it gives us the percentage of statements being covered by our tests. With this metric we can evaluate if every line of code is running.
Well, this is not entirely true, as Line coverage is a metric by itself. If a statement is composed by several lines, you can guess the difference
between line and statement coverage.</p>
<h3 id="branch-coverage">Branch coverage</h3>
<p>This type of coverage evaluates if all branches are executed. If we have an <em>if</em> statement without the <em>else</em> clause, then branch coverage would never be
100% because the code is missing the <em>else</em> clause. On the other hand if we have one <em>if then else</em> statement, the branch coverage would be equal to the
statement coverage.</p>
<h3 id="loop-coverage">Loop coverage</h3>
<p>This metric can be important because bugs are commonly found in while loop boundary conditions. It specifies that we execute each loop 0 times, once,
and more than once.</p>
<hr />
<p>There are many <a href="http://www.kaner.com/pdfs/negligence_and_testing_coverage.pdf">more metrics out there</a>, including <a href="http://en.wikipedia.org/wiki/Modified_condition/decision_coverage">MC/DC</a>
(Modified Condition/Decision Coverage) used heavily in avionics software.</p>
<blockquote>
<p>Having a good coverage score is good but it’s only an indicator that our tests are running all the code, not that the code does everything
according to the specs.</p>
</blockquote>
<p><a href="http://hugocaracol.github.io/testing/code-coverage/">Code coverage</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on February 04, 2013.</p>http://hugocaracol.github.io/testing/software-testing2013-01-30T23:10:00+00:002013-01-30T23:10:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Today I started <a href="https://www.udacity.com/course/cs258">Software Testing class</a> on Udacity.<br />
In the first unit Prof. John Regehr talked about a specification issue (feets vs meters) in the Mars Climate Orbiter. I remember hearing about that before
but I got curious and found <a href="http://www.computerworld.com/s/article/9183580/Epic_failures_11_infamous_software_bugs">this list of software bugs</a>.</p>
<h3 id="class-summary">Class Summary</h3>
<blockquote>
<p>When writing software, destruction can be just as valuable as creation. Learn how to catch bugs and break software as you discover different testing methods that will help you build better software.</p>
</blockquote>
<p><a href="http://hugocaracol.github.io/testing/software-testing/">Software Testing</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on January 30, 2013.</p>http://hugocaracol.github.io/python/class-inheritance-check-with-mro2013-01-12T13:41:00+00:002013-01-12T13:41:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Consider the following code:</p>
<div class="highlight"><pre><code class="python"><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="n">B</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">D</span><span class="p">(</span><span class="n">C</span><span class="p">):</span>
<span class="k">pass</span>
</code></pre></div>
<p>As you can see, D derives from C, which derives from B, which derives from A, which derives from object. In this example is very clear to see the relations between the classes. But usually, things get more complicated and you can’t figure out in a second how the classes fit together. Specially if there’s no documentation!</p>
<p><strong>MRO</strong> stands for <strong>Method Resolution Order</strong> and defines the path to follow when calling methods belonging to super classes.</p>
<!-- more -->
<p>Use <your_class>.__mro__ to print a tuple which describes the order of the method resolution.</p>
<p>Let’s print out D.__mro__</p>
<div class="highlight"><pre><code class="python"><span class="o">>>></span> <span class="n">D</span><span class="o">.</span><span class="n">__mro__</span> <span class="c"># try also: [x.__name__ for x in D.__mro__]</span>
<span class="p">(</span><span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">__main__</span><span class="o">.</span><span class="n">D</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">C</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">B</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">A</span><span class="s">'>, <type '</span><span class="nb">object</span><span class="s">'>)</span>
</code></pre></div>
<p>How about if we change the defintion of D.</p>
<div class="highlight"><pre><code class="python"><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">B</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">C</span><span class="p">(</span><span class="n">B</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">D</span><span class="p">(</span><span class="n">C</span><span class="p">,</span> <span class="n">B</span><span class="p">):</span>
<span class="k">pass</span>
</code></pre></div>
<div class="highlight"><pre><code class="python"><span class="o">>>></span> <span class="n">D</span><span class="o">.</span><span class="n">__mro__</span>
<span class="p">(</span><span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">__main__</span><span class="o">.</span><span class="n">D</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">C</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">B</span><span class="s">'>, <class '</span><span class="n">__main__</span><span class="o">.</span><span class="n">A</span><span class="s">'>, <type '</span><span class="nb">object</span><span class="s">'>)</span>
</code></pre></div>
<p>Python first looks in D, then in C, then in B, then in A and then in object.</p>
<p>But beware! Change the definition of D - <em>class D(B, C):</em> - and check its __mro__.</p>
<p><a href="http://hugocaracol.github.io/python/class-inheritance-check-with-mro/">Class inheritance check with __mro__</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on January 12, 2013.</p>http://hugocaracol.github.io/python/decorators-in-python2013-01-08T20:56:00+00:002013-01-08T20:56:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Today I’ll turn you into a decorator. Let’s build a house and decorated it.
First things first… the house. In my world it’s very simple to build a house, just call the function build_house().
After the “construction”, in order to live there, we need to turn it cozier. For this effect we can call a decorator to shine things up.
When she arrives and sees the house she says: I’m going to decorate it with paitings on the wall. With no further delay she: @decorate(“paitings”).</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">decorate</span><span class="p">(</span><span class="n">thing</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">_decorate</span><span class="p">(</span><span class="n">func_name</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__decorate</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">ret</span> <span class="o">=</span> <span class="n">func_name</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">if</span> <span class="n">thing</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">ret</span> <span class="o">+=</span> <span class="s">" decorated with </span><span class="si">%s</span><span class="s"> on the wall"</span> <span class="o">%</span> <span class="n">thing</span>
<span class="k">return</span> <span class="n">ret</span>
<span class="k">return</span> <span class="n">__decorate</span>
<span class="k">return</span> <span class="n">_decorate</span>
<span class="nd">@decorate</span><span class="p">(</span><span class="s">"paintings"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">build_house</span><span class="p">():</span>
<span class="k">return</span> <span class="s">"This is a house"</span>
</code></pre></div>
<div class="highlight"><pre><code class="python"><span class="o">>>></span> <span class="k">print</span> <span class="n">build_house</span><span class="p">()</span>
<span class="n">This</span> <span class="ow">is</span> <span class="n">a</span> <span class="n">house</span> <span class="n">decorated</span> <span class="k">with</span> <span class="n">paintings</span> <span class="n">on</span> <span class="n">the</span> <span class="n">wall</span>
</code></pre></div>
<!-- more -->
<p>##Basic Usage
Decorators are used to change a function behaviour not by altering its code but by trapping the entry and exit points. Suppose you want to log the execution time of a function or make it return different values according to additional rules. The code to do that fits perfectly in a decorator.
## Explanation
When a function is decorated, it is run inside the decorator. This way, it is granted to the decorator the power to do the things described earlier.
###Functions return functions
Functions are first class objects. That means a function is indeed an object. Objects can be passed as arguments and be returned by functions. Therefore, functions can consume functions and return functions (replace the word functions with objects and you get a true statement as well).</p>
<p>The statement <em>functions can return functions</em> implies that an argument of a function can be a function and/or a function can be defined inside a function.</p>
<p>I’m getting sick of writing the word <em>function</em> but let’s continue with a piece of code.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="k">print</span> <span class="s">'inner'</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="n">fn</span> <span class="o">=</span> <span class="n">outer</span><span class="p">()</span>
<span class="n">fn</span><span class="p">()</span> <span class="c"># prints inner</span>
</code></pre></div>
<p>The code above defines the function inner inside outer. When we run outer(), the fn variable becomes equal to the function inner. </p>
<blockquote>
<p>The line 4 returns the inner function without calling it (did you notice the lack of parentheses?).</p>
</blockquote>
<p>Now if we call fn by adding parentheses () we get the execution of inner.</p>
<p>This leads us to…</p>
<h3 id="closures">Closures</h3>
<p>From Wikipedia…
>A closure—unlike a plain function pointer—allows a function to access those non-local variables even when invoked outside of its immediate lexical scope.</p>
<p>This means that our inner function can “remember” non-local variables, in our case, variables defined in outer function.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="n">x</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="k">print</span> <span class="s">'Got value </span><span class="si">%i</span><span class="s"> from non-local'</span> <span class="o">%</span> <span class="n">x</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="n">fn</span> <span class="o">=</span> <span class="n">outer</span><span class="p">()</span>
<span class="k">del</span> <span class="n">outer</span> <span class="c"># deletes object outer</span>
<span class="n">fn</span><span class="p">()</span> <span class="c"># 'Got value 1 from non-local'</span>
</code></pre></div>
<p>Even deleting the function outer, fn() still prints x variable.</p>
<h3 id="functions-as-arguments">Functions as arguments</h3>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">run_me</span><span class="p">():</span>
<span class="k">print</span> <span class="s">"I'm running baby"</span>
<span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">()</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">outer</span><span class="p">(</span><span class="n">run_me</span><span class="p">)</span>
<span class="k">del</span> <span class="n">run_me</span> <span class="c"># deletes run_me function</span>
<span class="n">r</span><span class="p">()</span> <span class="c"># "I'm running baby"</span>
</code></pre></div>
<p>On line 9, r gets function inner, returned by outer, which calls run_me (closure), which prints “I’m running baby”.</p>
<h3 id="decorate-me">Decorate me!</h3>
<p>Now let’s try some decorator magic.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">()</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="nd">@outer</span>
<span class="k">def</span> <span class="nf">run_me</span><span class="p">():</span>
<span class="k">print</span> <span class="s">"I'm running baby"</span>
<span class="n">run_me</span><span class="p">()</span> <span class="c">#call run_me and prints the next 2 lines</span>
<span class="n">Before</span> <span class="n">running</span> <span class="n">the</span> <span class="n">function</span>
<span class="n">I</span><span class="s">'m running baby</span>
</code></pre></div>
<p>This is a simple example of a decorator. To apply the decorator to a function you use @ symbol followed by the decorator name. Here you no longer need to use ‘r = outer(run_me)’ and ‘r()’. Much simpler!</p>
<p>How about if we change the definition of function run_me and add a parameter name to it? Will the decorator still works? Let’s try!</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">()</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="nd">@outer</span>
<span class="k">def</span> <span class="nf">run_me</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"I'm running </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">name</span>
<span class="n">run_me</span><span class="p">(</span><span class="s">"Hugo"</span><span class="p">)</span> <span class="c"># error below</span>
<span class="ne">TypeError</span><span class="p">:</span> <span class="n">inner</span><span class="p">()</span> <span class="n">takes</span> <span class="n">no</span> <span class="n">arguments</span> <span class="p">(</span><span class="mi">1</span> <span class="n">given</span><span class="p">)</span>
</code></pre></div>
<p>We get an error saying function inner takes no arguments but we tried to pass the name “Hugo”. Sadly it doesn’t work. Happily we know how to fix it :)</p>
<p>To fix it we add a parameter ‘name’ to the function inner and then add the argument to the function call on line 4 below.</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="nd">@outer</span>
<span class="k">def</span> <span class="nf">run_me</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"I'm running </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">name</span>
<span class="n">run_me</span><span class="p">(</span><span class="s">"Hugo"</span><span class="p">)</span> <span class="c"># now it works</span>
<span class="c">#Before running the function</span>
<span class="c">#I'm running hugo</span>
</code></pre></div>
<p>Ok, so far so good. But what happens if I change the definition of the function run_me and add some more parameters? It will FAIL! Let’s make the decorator more generic so we don’t have to worry too much.</p>
<h3 id="args-and-kwargs">*args and **kwargs</h3>
<p>*args and **kwargs are conventional ways to get the arguments passed to a function.</p>
<p>For more info on *args and **kwargs please refer to one of the many posted questions on <a href="http://stackoverflow.com/questions/3394835/args-and-kwargs">Stackoverflow</a> or <a href="http://docs.python.org/2/tutorial/controlflow.html#arbitrary-argument-lists">Python docs</a>.</p>
<p>As we want to abstract from the number of arguments passed to the function inner we replace ‘def inner(name):’ with ‘def inner(*args, **kwargs):’ and ‘func(name)’ with ‘func(*args, **kwargs)’. </p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="nd">@outer</span>
<span class="k">def</span> <span class="nf">run_me</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"I'm running </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">name</span>
<span class="n">run_me</span><span class="p">(</span><span class="s">"Hugo"</span><span class="p">)</span> <span class="c"># now it works</span>
<span class="c">#Before running the function</span>
<span class="c">#I'm running hugo</span>
</code></pre></div>
<h3 id="final-touch---passing-arguments-to-the-decorator">Final touch - passing arguments to the decorator</h3>
<p>This step is only required if you want to add some salt to the decorator, changing the way it works when certain arguments are passed to it. A lot can be “decorated” without this step.</p>
<p>The decorator function now has a parameter that indicates if we run it in debug mode or not. To get this, we add another “level” of functions. As you can see from the code below the decorator is composed by the function outer (with takes the debug argument), the function _outer (which takes the ‘function’ argument) and the function inner (which takes the arguments to the function to be decorated).</p>
<div class="highlight"><pre><code class="python"><span class="k">def</span> <span class="nf">outer</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">_outer</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">inner</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">if</span> <span class="n">debug</span><span class="p">:</span>
<span class="k">print</span> <span class="s">"Before running the function"</span>
<span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">inner</span>
<span class="k">return</span> <span class="n">_outer</span>
<span class="nd">@outer</span><span class="p">(</span><span class="bp">False</span><span class="p">)</span> <span class="c"># decorated is NOT in debug mode</span>
<span class="k">def</span> <span class="nf">run_me</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"I'm running </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">name</span>
<span class="n">run_me</span><span class="p">(</span><span class="s">"Hugo"</span><span class="p">)</span>
<span class="c"># I'm running Hugo</span>
</code></pre></div>
<p>This concludes the post.</p>
<h4 id="disclaimer">Disclaimer</h4>
<p>Much of the ideas described here came from Simeon Franklin’s <a href="http://simeonfranklin.com/blog/2012/jul/1/python-decorators-in-12-steps/">post on decorators</a>.
This post is available at <a href="http://www.codeproject.com/script/Articles/BlogFeedList.aspx?amid=10549383" rel="tag">CodeProject</a>.</p>
<p><a href="http://hugocaracol.github.io/python/decorators-in-python/">Decorators in Python</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on January 08, 2013.</p>http://hugocaracol.github.io/github/migration-to-github-pages2013-01-04T23:07:00+00:002013-01-04T23:07:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com
<div class="highlight"><pre><code class="bash"><span class="nv">$ </span>ruby -rubygems -e <span class="s1">'require "jekyll/migrators/wordpressdotcom"; Jekyll::WordpressDotCom.process("wordpress.xml")'</span>
</code></pre></div>
<p><a href="http://hugocaracol.github.io/github/migration-to-github-pages/">Migration to github pages</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on January 04, 2013.</p>http://hugocaracol.github.io/random/portuguese-dictionary2012-12-28T00:00:00+00:002012-12-28T00:00:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Today I played with beautifulsoup and wrote a script to query a portuguese online dictionary.</p>
<div class="highlight"><pre><code class="bash"><span class="nv">$ </span>./porto-dict <word>
</code></pre></div>
<div class="highlight"><pre><code class="bash"><span class="nv">$ </span>./porto-dict.py olá
</code></pre></div>
<p>olá [ó] (interjeição)<br />
1. usada como saudação ou chamamento<br />
2. exprime espanto</p>
<p>Code is available on <a href="https://github.com/hugocaracol/porto-editora-dict">Github</a>.</p>
<p><a href="http://hugocaracol.github.io/random/portuguese-dictionary/">Portuguese dictionary</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on December 28, 2012.</p>http://hugocaracol.github.io/random/nokogiri2012-12-27T00:00:00+00:002012-12-27T00:00:00+00:00Hugo Santoshttp://hugocaracol.github.iohugonsantos@gmail.com<p>Today I found <a href="http://nokogiri.org/">Nokogiri</a>.
>Nokogiri is an HTML, XML, SAX, and Reader parser. Among Nokogiri’s many features is the ability to search documents via XPath or CSS3 selectors.</p>
<p>But the description above is not the reason of this post. This is:</p>
<blockquote>
<p>XML is like violence - if it doesn’t solve your problems, you are not using enough of it.</p>
</blockquote>
<p><a href="http://hugocaracol.github.io/random/nokogiri/">Nokogiri</a> was originally published by Hugo Santos at <a href="http://hugocaracol.github.io">Stack Data Structure</a> on December 27, 2012.</p>