CoderOnCode

Ethereum Zero to Hero: Developing Our First Contract

Sun, 24 Dec 2017 14:21:37 -0500

This is the third part of our Ethereum Zero to Hero guide. If you have not read the previous post, I highly recommend checking it before moving forward.

In this post we are going take were we left off and start developing a basic Smart Contract and learn more about solidity and how the contracts work.

One of the most interesting qualities of the Ethereum blockchain is the ability create digital tokens. Tokens are digital assets that can represent anything from coins, loyalty points to actual objects in the physical world. Tokens can also be used to interact with other smart contracts.

Tokens are sold and traded as a cryptocurrency inside the ethereum network and allows the user to pay for smart contract transactions fees, for our first contract we are going to create the minimum viable contract needed to deploy on our private network.

A Word on ERC20

Often when dealing with Tokens, you will see the word ERC20 being referenced. ERC20 is a token standard that describes the functions and events that an Ethereum token contract has to implement, by following this standard we guarantee that tokens will be instantly compatible with the ethereum wallet and any other client or contract that uses the same standard.

In order for a token to be ERC20 complaint it must implement the following:

For more details on ERC20 standard view the following links:

The Ethereum Wallet

The Ethereum Wallet is a gateway to decentralized applications on the Ethereum blockchain. It allows you to hold and secure ether and other crypto-assets built on Ethereum, as well as write, deploy and use smart contracts.

Up to this point, we have worked directly on the Geth console, and while I highly recommend Ethereum developers to become familiar and acquainted with Geth in order to understand some of the Ethereum inner-workings in order to move faster and work smarter, we will switch to the official Ethereum wallet:

Make sure to download and install the ethereum wallet before continuing.

Connecting the wallet to our private testnet

By default the Ethereum Wallet will attempt to connect to the main network, as well it will allow you to connect to one of the test networks.

For our purposes we want it to automatically start connected to our testnet, so do the following before opening the wallet:

DISCLAIMER: The following operations will override any local information for the main network, this could potentially result in the loss of ether, be careful and understand the commands you are running.

Stop the running geth node
Run the following command

geth --datadir ./TestNetData --identity "HeroNode1" --ipcpath ~/Library/Ethereum/geth.ipc --networkid 24 --maxpeers 1 console

Now we can open our ethereum wallet, and it should automatically connect to the test network, what we did was restarting our Geth node, and setting the IPC (Inter-process Communication) pipe in the same place the wallet has its default, by doing so the wallet will directly establish a connecting to our Geth node instead of creating one to the main network.

Make sure the wallet is displaying the name of the network as Private-Net before continuing.

Minimum Viable Token

As we learned in the first post of this series, contracts are programmed using the Solidity language; there are several tools available for compiling, debugging and writing solidity contracts, for the time being, we are going to use solidity contract editor that is shipped with the ethereum wallet.

Here is the code for our minium viable token:

Now grab the source code from above and paste it in the Solidity Contract Source Code field. The wallet will automatically check if the code compiles and if it does, you will see a “pick a contract” dropdown on the right like so:

One interesting thing to notice is that there is also one constructor parameter, in this case, an initial supply of Hero Tokens, that will be an issue on contract creation.

If you keep scrolling down, you’ll see an estimate of the computation cost of that contract, and you can select a fee on how much ether you are willing to pay for it. Don’t forget to set the initial number of tokens we will issue, now go ahead and click the “deploy” button.

Next, you will see a confirmation dialog like the following:

Once the contract has been submitted, you should see it in the latest transactions below, waiting for confirmation. Usually, it should take around 30 seconds for the contract to be confirmed but, at this point, we could wait and wait, and nothing would happen.

This is because there are no other nodes in the network and no one is mining, and ethereum much like Bitcoin right now works as a Proof of Work network, I will not go into the details, but you can read the following article for a thorough explanation:

Explaining blockchain — how proof of work enables trustless consensus

Let’s get our contract verified by jumping into our node console and typing:

miner.start()

It will take about 12 transactions for the contract to be confirmed, once that is done we can stop the miner by typing:

miner.stop()

Looking at the latest transactions our contract should be now verified, we can confirm that by going into the “Contracts” tab and we should see our newly created contract.

Click on it to go to its page. Click “copy address” and paste the contract address into a text editor, you will need it shortly.

If you click your wallet address, there is no indication of those tokens being there, did something went wrong? Well no, Wallets only track tokens it knows about, and we have not told our wallet to track our brand new token.

Tracking Tokens

Go to the Contracts page
Click the Watch Token Button
A popup will show up, copy the address of the contract as well make sure to enter all the details
Click Ok

Now if you go back to your wallet you should see that is tracking tokens as the screenshot below.

Congratulations! You successfully created your first token.

Next Steps

While we successfully created a basic token, right now is not every useful. In the next post, we will improve our HeroToken contract and add more advanced functionality.

Ethereum Zero to Hero: Setting a Development Environment

Sun, 24 Dec 2017 14:21:31 -0500

This is the second part of our Ethereum Zero to Hero guide. If you have not read part, I highly recommend it before jumping ahead.

In this post, we are going to set up a basic private test-net environment for ethereum development and experimentation.

Step 0: Requirements

You are using MacOS
You have a basic understanding of software development
You know what Ethereum is and understand it is basic usage.
You have a basic understanding of MacOS Terminal usage.
You have homebrew installed

Step 1: Setting Test-Net

While we are learning the in’s and out’s of Ethereum development, we probably don’t want to test against the real production network with real ether.

There are several Ethereum tests-nets out in the wild for this purpose Rinkeby and Morden for example; however interacting with this networks still requires for us to acquire ethereum either by mining, or getting ether from other users.

While we will eventually work with the public test networks, for beginners like us that are just starting out, that is far from ideal, and it can be difficult to get significant amounts of ether.

A better option is setting up our private testnet, that we can bend and twist; as well it will allow us to gain a deeper understanding of the inner workings of the Ethereum network.

Let’s GETH Going

We will need to install the geth, a go CLI client that will allow us to run a full ethereum node locally.

brew tap ethereum/ethereum brew install ethereum

We can verify that everything installed correctly by typing:

geth version

Which should give you an output similar to the following:

Step 2: Genesis Block

To get our on private testnet started, we are going to need a Genesis Block. Every Blockchain requires a genesis block, which is essentially the configuration file for our blockchain.

The genesis block is the start of the blockchain - the first block, block 0, and the only block that does not point to a predecessor block. The protocol ensures that no other node will agree with your version of the blockchain unless they have the same genesis block, so you can make as many private testnet blockchains as you would like!

Create a project directory, for example ethereum_0hero
Open your preferred editor and create a file named HeroGenesis.json
Copy the following contents and save the file:

As you can see the file contents are in json and for the most part self-explanatory, but let’s go over the each one of the parameters quickly to clarify their purpose.

coinbase: The 160-bit address to which all rewards (in Ether) collected from the successful mining of this block has been transferred. This can be anything in the Genesis Block since the value is set by the setting of the miner when a new block is created.
timestamp: A scalar value equal to the reasonable output of Unix time() function at this block inception. A smaller period between the last two blocks results in an increase in the difficulty level and thus additional computation required to find the next valid block.
difficulty: It defines the mining Target, which can be calculated from the previous block’s difficulty level and the timestamp. The higher the difficulty, the statistically more calculations a Miner must perform to discover a valid block. This value is used to control the Block generation time of a Blockchain, keeping the Block generation frequency within a target range. On the test network, we keep this value low to avoid waiting during tests, since the discovery of a valid Block is required to execute a transaction on the Blockchain.
gasLimit: A scalar value equal to the current chain-wide limit of Gas expenditure per block. High in our case to avoid being limited by this threshold during tests.

Finally, the config parameters are there to ensure that certain protocol upgrades are available from the get go.

For a full explanation check this answer in stackexchange

Next, we will need to initialize our chain by running the following command:

geth --datadir ./TestNetData init HeroGenesis.json

Setting the data directory is important since otherwise we will override the default data directory for the real Ethereum network.

The output should return something similar to the following:

Step 3: Running a Local Node

Now that we create our initial chain we can run it by executing the following command:

geth --datadir ./TestNetData --identity "HeroNode1" --rpc --rpcport "8080" --rpccorsdomain "*" --port "30303" --nodiscover --rpcapi "db,eth,net,web3" --maxpeers 0 --networkid 24 console

This command does a few things:

Utilizes the Genesis block we previously created
It uses a custom data directory instead of the default
Sets the network id to 24 to prevent us from talking from noes from the main network
Disables peer discovery
Disables the network by setting up the maxpeers to 0
Launches the geth console so we can interact with the blockchain/node

The output should be something similar to the following:

At this point, you should be up and running, and in the geth console ready to do something, but what?

Well let’s switch attention to the following Warning:

WARN No etherbase set and no accounts found as default

We have node up and running, but it will not be much use without any accounts.

Step 4: Creating an account

Since we are already in the Geth console is easier to ahead and create an account directly from there, in the terminal type the following:

personal.newAccount()

The console will ask for a passphrase DO NOT LOSE THIS!, and return a has like the following:

0xe857331e4e3354bb72b3751cce419c8444e89e17

And let’s validate that the account exist by running:

> eth.getBalance("0xe857331e4e3354bb72b3751cce419c8444e89e17") 0

Now, let’s put some ether into that account.

Step 5: Let’s Geth Mining

Quick recap:

We have a working private network with a single node running locally
We create a test account that on said private network
We do not have any ether in that account, and we need to fix that

Now, while we could easily issue ether to our account using the Genesis block, let’s have some fun and mine Ether on our private network.

Open a new terminal tab and running the following command:

get attach ./<DirectoryName>/geth.ipc

This will connect our second terminal to the existing geth instance and open the Javascript console. Inside the console run the following:

miner.setEtherbase("0xe857331e4e3354bb72b3751cce419c8444e89e17") miner.start()

If we go back to our first terminal we should see the following happening:

Let’s go back to the second terminal and stop the miner and check our balance:

miner.stop() eth.getBalance('0xe857331e4e3354bb72b3751cce419c8444e89e17') 75000000000000000000

Congratulations! We have now created our private network and mined our few amount of ether, and we have a valid network that we can use to developer our Smart Contracts and DApps.

In the next post of the series, we will start developing with Solidity and Smart Contracts.

Ethereum Zero to Hero: Introduction

Sun, 24 Dec 2017 14:21:20 -0500

2017 has arguably been the year of Cryptocurrency, with Bitcoin being getting most of the spotlight; at the core of Bitcoin, we have the Blockchain.

Blockchain technology applications go way beyond digital currency, and one of the best examples is Ethereum which is a decentralized platform that runs smart contracts.

This allows developers to build enormously powerful decentralized applications, at this point there is still a lot of active development and innovation happening both around blockchain and ethereum.

The downside of all this constant innovation and development is that tutorials, documentation, and resources go out of date quickly, this has made it difficult for developers like me (or you) to get a solid footing when getting started.

This guide is not to mean to the end all or be all, rather a quick introduction that can get you started quickly and hopefully agnostic enough that won’t go out of date too fast. With that said, let’s get started by reviewing some core concepts:

Smart Contracts

Contracts live on the blockchain in an Ethereum-specific binary format (EVM bytecode).

A smart contract is a piece of software that resides on the Ethereum Blockchain. Like traditional contracts, smart contacts not only define the rules and penalties around an agreement but additionally the enforce those obligations.

Ethereum Virtual Machine

At the heart of it is the Ethereum Virtual Machine (“EVM”), which can execute code of arbitrary algorithmic complexity. In computer science terms, Ethereum is “Turing complete.”

This is the core and primary innovation behind the Ethereum project. Each participant of the Ethereum networks runs an instance of the virtual machine, and its purpose is to execute the smart contracts in a completely isolated environment, meaning no access to Network, Filesystem or other processes.

Gas

Gas is a concept unique to the ethereum platform and is way to limit the resources available to a given smart contract. For every instruction executed in the EVM, there is a fixed Gas cost associated with it.

Solidity

Solidity is a contact-oriented, high-level language for implementing smart contracts. The syntax resembles javascript and is influenced by languages like C++ and Python, and it compiles directly to EVM assembly.

Blockchain

“The blockchain is an incorruptible digital ledger of economic transactions that can be programmed to record not just financial transactions but virtually everything of value.” - Don & Alex Tapscott, authors Blockchain Revolution (2016)

The best way to think about the blockchain is as decentralized immutable database or ledger, that can permanently store any type data.

The potential business applications for this technology are still being discovered and experimented with, but there are tons of examples online, to mention a few we have:

Crowdfunding
Governance
File Storage
Protection of Intellectual Property
Identity Management
Property Registration

In the next entry in this series we will set up a local development environment for creating our first smart contract.

5 Trends To Watch Out On 2018

Mon, 18 Dec 2017 20:13:44 -0500

With 2018 around the corner is prudent to take some time and see what new and existing trends will dominate 2018.

Blockchain

In 2017 we saw many of the Silicon Valley giants experiment or even introducing their own blockchain implementations, IBM been the most significant one by collaborating with food distributors, government regulation agencies and banks for leveraging blockchain.

Why it matters

With applications ranging from cryptocurrency, to smart properties, to identity like passports; we have only started scratching the surface of what Blockchain technology can offer and in 2018 it won’t be a surprise if we see a rise in the creation and adoption of blockchain based application.

In fact, in 2017 Amazon, Oracle, Microsoft and IBM all started experimenting and working on their own blockchain platforms, and in 2018 we will see more companies joining the bandwagon across a myriad of different fields.

Conversational Interfaces

Conversational interfaces are nothing new, and while 2017 saw the birth of countless chatbots in all number of platforms, 2018 will be the year were conversational interfaces will continue growing and gain significant maturity.

Why it matters

Conversation interfaces vary from voice recognition platforms like Siri, Alexa and Google Home; to hundreds of bots on platforms like Facebook, Telegram and Slack; but they all have one thing in common, they are slowly changing the way we interact with software.

Regardless if its voice or text, in 2018 conversation interfaces will continue to provide an improved and more personalized user experience.

AI will take over …

… but not in the way you think. In 2018 we will see a raise in solutions and applications augmented by the use Machine Learning/AI.

Why it matters

For many AI seems to be an all or nothing proposition, the reality is that AI is already here and contributing in unexpected ways. In 2018 we will see the transition from AI being somewhat of a buzzword and a nice to have, to an absolute necessity for startups and businesses.

In 2017, there was a significant number of businesses using AI to drive financial and operation decisions, not only this trend will continue gaining steam but we will also see AI application in less obvious fields.

The Rise of Backends As A Service (BaaS)

Backend as a service, while typically is not reference by that name is a trend that has been on the rise in 2017 and will continue gaining steam during 2018.

Why it matters

BaaS turns traditional parts of the backend into a third party service that is consumed by other applications, some good examples of BaaS are push notifications, file storage and sharing, integration with social networks and messaging and chat, just to mention a few.

One trend that I’m expecting to see more of in 2018 is what Shopify is doing with its Storefront API, in which Shopify essentially becomes a headless backend for a fully customizable shopping experience.

Augmented Reality

AR and mobile seem to be the perfect marriage and with the release of ARKit by Apple and ARCore by Google, 2018 is poised to be the year where we will see mainstream adoption of AR.

Why it matters

While there has been a lot of hype surrounding Virtual Reality in the past couple years, there is still ways to go before it becomes commonplace and widely adopted. On the other hand Augmented Reality, might become mainstream in 2018.

What to Read Next

Photo by Eddie Kopp on Unsplash

You Should Learn Functional Programming in 2017

Sun, 23 Apr 2017 16:53:15 -0400

Functional programming has been around for a very long time, starting in the 50’s with the introduction of the Lisp programming language; and if you been paying attention in the last two years languages like Clojure, Scala, Erlang, Haskell and Elixir have been making a lot of noise and getting tons of attention.

But what is functional programming, why is everyone going crazy about it and why the heck are not more people using it? In this post, I will attempt to answer all of those questions and hopefully pique your interested on the idea of functional programming.

A Brief History of Functional Programming

As we said before functional programming started back in the 50’s with the creation of Lisp to run in the IBM700/7000 series of scientific computers. Lisp introduced many paradigms and features that we associate with functional programming now days, and while we can call Lisp the granddaddy of functional programming we can even further to a common root between all functional programming languages, Lambda Calculus.

This is by far the most interesting aspect of functional programming; all functional programming languages are based on the same simple mathematical foundation, Lambda Calculus.

Lambda calculus is Turing complete, that is, it is a universal model of computation that can be used to simulate any single-taped Turing machine.[1] Its namesake, the Greek letter lambda (λ), is used in lambda expressions and lambda terms to denote binding a variable in a function. – Wikipedia

Lambda Calculus is a surprisingly simple yet powerful concept. At the core of lambda calculus we have two concepts:

Function abstraction, which is used to generalize expressions through the introduction of names (variables).
Function application, which is used to evaluate the generalized expressions by giving names to particular values.

Let’s take a look at an example, a single argument function f that increments the argument by one:

f = λ x. x+1

Let’s say we want to apply the function to the number 5; then the function can be read as follows:

f(5) => 5 + 1

Functional Programming Fundamentals

Enough Math, for now, lets take a look at the features that make Functional programming a powerful concept:

First-Class Functions

In functional languages, functions are first-class citizens this means that a functions can be stored in a variable, for example in elixir that would look like this:

double = fn(x) -> x * 2 end

And as such we can easily call the function as follows:

double.(2)

High-Order Functions

A high order function is defined as a function that takes one or more functions as arguments and/or that returns a new function. Let’s use our double function again to exemplify the concept:

double = fn(x) -> x * 2 end Enum.map(1..10, double)

In this case Enum.map takes an enumerable –a list– as the first argument and the function we just defined as the second; and applies the function to each element in the enumerable the result being:

[2,4,6,8,10,12,14,16,18,20]

Immutable State

In functional programming languages state is immutable, this means that once a variable has been bound to a value they cannot be redefined, this has the nice advantage of preventing side effects and race conditions; making concurrent programming much much easier.

As before let’s use Elixir to illustrate this concepts:

iex> tuple = {:ok, "hello"}
{:ok, "hello"}
iex> put_elem(tuple, 1, "world")
{:ok, "world"}
iex> tuple
{:ok, "hello"}

In the example our tuple will never change values, in the third line put_elem is returning a completely new tuple without modifying the values of the original.

I won’t go into further details because this post is not an introduction to lambda calculus, computing theory or even functional programming; if you would like me to dig deeper on either subject drop me a message on the comment section. For now our take away from this section is the following:

Functional Programming has been around for a long time (the early 50’s)
Functional Programming is based on mathematical concepts, specifically Lambda Calculus
Functional Programming was deemed too slow compared to imperative languages
Functional Programming is making a comeback.

Functional Programming Applications

As software developers we are living in exciting times, the promise of the cloud is finally here, and with it, an unprecedented amount of computer power is available to every single one of us. Unfortunately, with there also came demands of scalability, performance, and concurrency.

Objected Oriented Programming is simply not cutting it anymore, specially when it comes to concurrency and parallelism; trying to add concurrency and parallelism to this languages adds lots of complexity and more often than not leads to over engineering and poor performance.

Functional programming in the other hand is already well suited for these challenges, Immutable state, Closures, and High order functions, concepts that lend themselves very well for writing highly concurrent and distributed applications.

But don’t take my word for it, you can find enough proof by looking at the technological feeds of startups like WhatsApp and Discord:

WhatsApp was able to support 900 million users with only 50 engineers in their team by using Erlang.
Discord in similar fashion handles over a million requests per minute using Elixir.

These companies and teams are capable of handling this massive growth thanks to the advantages of functional programming, and as functional programming gains more and more traction; I strongly believe this stories like the WhatsApp and Discord will become more and more common.

For that reason functional programming needs to be a must on every developer’s toolbox of knowledge, you need to be ready to build the next generation of applications that will serve the next billion of users; and heck if that wasn’t enough trust me functional programming is really fun, just take a look at Elixir:

Finally, If you have any comments, corrections or would you like to know more about the subject don’t hesitate to leave a comment just below.

Neural Networks Without a PhD: Topologies

Sun, 16 Apr 2017 15:25:04 -0400

Topology of a neural network refers to the way the Neurons are connected, and it is an important factor in network functioning and learning. A common topology in unsupervised learning is a direct mapping of inputs to a collection of units that represents categories (e.g., Self-organizing maps). – Springer

In our previous post we learned about the components that form a neural network:

Input Nodes
Neurons
Output Nodes

as well that those elements will be organized in the following layers:

Input Layer
Hidden Layers
Output Layer

However, it still not very clear what the process for organizing and connecting those neural networks is; how do we determine how many hidden layers we need to create? How do we connect those neurons to the inputs, outputs and to each other? How do we know how many output nodes we need to create?

How to organize our Neural Network

What we need is a way to represent how neurons are connected in order to form a network, a Neural Network Topology. The topology of a neural network plays an essential role in its function and performance.

An important thing to notice is that multiple topologies can be used to learn the same set of data, and they are even likely to produce similar results; and as such there is not ‘best’ topology for a single neural network; that being said topology can greatly impact the amount of time required for a neural network to learn data as well its accuracy when classifying new data.

One a approach to topology selection is to simply make it a trial and error process, where we manually tweak the Neural network by modifying things like the number of hidden layers, connections between nodes, and so on. Taking this approach would limit us greatly as there are many possible permutations even for a simple neural network; to complicate matters even further, Neural networks are not limited to the relatively simple Feed-Forward layout we have seen before, to mention a few:

Feed-forward
Recurrent
Long Short Term Memory
Jordan
Elman
Hopfield

So outside of very simple applications, known problems or learning exercises manual topology selection is not a viable option, in order to find the ideal topology we have to let our neural network to learn the topology from the training data as well. The goal is to find a topology for our neural network that **minimizes the the error on new data. **

One of most novel approaches for this objective is the use of genetic programming.

Genetic Programming and Neuro Evolution

Genetic algorithms are based on the evolutionary principles of the natural selection, mutation and survival of the fittest; a genetic algorithm will approach a particular problem by generating a large number of potential solutions and finding the solutions with the best fitness score.

Without going too deep into the weeds, a fitness score is a method to measure how close was the solution to known result or to a expected metric; in the case of a neural network where we have training data that we can use as reference point.

The Genetic algorithm will tweak and mutate our neural network generation after generation until the desired fitness score is achieved or until we reach the maximum number of iterations.

Disclaimer: The above is a oversimplified description of how genetic algorithms work

I highly recommend, The Handbook of Neuroevolution through Erlang for an amazing breakdown of the subject, even if you are not into Erlang the author makes a terrific job on breaking down the concepts and introduction genetic algorithms.

Summary

I’m important that we understand how topology, that is the number of layers and how they are connect of our neural networks can impact their effectiveness; there are many existing types of artificial neural networks some best suited to particular tasks of applications, and is important to familiarize ourselves with the most common types.

Additional, in order to find the optimal topology one can use Genetic algorithms in order to try to evolve the ‘perfect’ topology for a given application.

There is still a lot more to learn and go, but feel free to leave a comment if you find any factual errors or if you think something needs more clarification.

Vim Is The Perfect IDE

Sun, 16 Apr 2017 11:55:59 -0400

Over the years I’ve jumped back and forth between many code editors, IDEs and tools; but it seems that somehow I always end up coming right back to VIM, and not only for programming – guess which markdown editor I’m using to write this post.

I’ve have tried Atom, SublimeText, TextMate, Eclipse, Visual Studio, and most of the Jetbrains products, I’m constantly tweaking and looking for a better setup, however Vim always feels like home to me; and I’m to the point now where I rarely use IDEs – exception being messy and complex projects where IDEs can do a lot of heavily lifting (yes, Magento I’m talking about you.)

But other than that Vim is my default Ruby, Elixir, Python, PHP IDE and as well the main tool that I use for writing drafts and books.

What it look like

The Setup

So how does this magical tool work? Is all out of the box right? right? Well no, as with all the worthwhile things in life there is a bit of effort involved on getting the Vim setup just like I wanted it. Fortunately, is far from custom and is mostly the right combination of plugins.

You can find my current Vim configuration and dot files in its corresponding Github repository feel free to fork it and give a shot.

We are in particular interested in the vimrc file, let’s break it down:

Each plugin in this setup is separated in the following categories:

Utility

This kind of a miscellaneous category and is comprised of plugins used to enhance or change the behaivour of core vim; the most useful important ones are:

Nerdtree: It gives you easy access to the file system in the form of a directory tree on the left side of the screen, as well provides shortcuts for filesystem manipulation(create, delete, move files and directories)
Tagbar: Quick tag browser for the current file, a must have if you are using any kind of ctags like exuberant-tags.
FZF: Fuzzy finder, another handy utility for finding files and commands.
Neocomplete: Vim Autocomplete on steroids.

Generic Programming Support

These plugins fall directly on the category of programming and are used my all or most of the programming languages that I currently have setup:

Exuberant-Ctags: tags are named definitions of classes, functions, abstract types and so on; adding support to Vim gives you some of that ‘magic’ IDE code navigation functionality.
Syntastic: THE syntax checking plugin for Vim, if you are familiar with the way that code inspections work on Jetbrains and similar IDEs, syntastic will make feel right at home.
Vim-autoclose: Automatically closes a character that could/should have a matching closing counterpart, like () “” [] {} and so on.

Markdown/Writing

As I mentioned Vim is my go to editor for drafting new posts be it books, blogs or random angry letters. From this particular section only language tools deservers a special shot-out as it makes for a great Grammar checker directly from inside Vim.

Erlang/Elixir/PHP/Elm Support

When it comes down to the individual language support there isn’t really much to highlight other than I’ve tried (and somewhat failed) to keep the plugins to a minimum and focus only on the essential language support.

So far elixir is winning the battle in terms of plugins as I’ve added additional functionality like the ability to run tests and generate content from inside Vim, I have yet to decide if I’m keeping all the plugins for it.

Git Support

Standard git support that I’m afraid I rarely use, I find myself going directly back to the shell and doing the git workflow outside of Vim, so I’m open to suggestions and to hear what everyone else is using in terms of setup.

Themes and Interfaces

Ok this is a big one but mostly because I keep forgetting to remove unused themes and colorschemes, let’s highlight the important ones:

Vimarline: Lean and mean status/tabline for Vim; it also looks cool as fuck.
Vim-Devicons: Because not only atom gets all the fancy icons on the sidebar, highly recommended to if you are using nerdtree.

The remaining parts of the configuration file are either plugin configuration or personal key re-mappings that I use for quality of like; I’ve done my best to document and segment each section so it should be easy enough to understand what each setting is doing.

For anyone getting starting with Vim, I do want to bring special attention to the following:

" Disable arrow movement, resize splits instead. if get(g:, 'elite_mode') nnoremap <Up> :resize +2<CR> nnoremap <Down> :resize -2<CR> nnoremap <Left> :vertical resize +2<CR> nnoremap <Right> :vertical resize -2<CR> endif

As there is no quickest way to force one-self to use the home row for navigation.

Final Remarks

I hope that for some of you this post has been helpful for some, and I’m far from done with my current setup, in that sense this is very much a toy that I continue playing on a nearly daily basis; now that being said and in order to get back to the original premise of the post; it is indeed a very powerful toy, specially when combined with other tools like Tmux, here is a sneak peak of my Elixir ‘IDE’ powered by Vim and Tmux:

If you want to know more about my setup, or want to share yours please leave a comment below.

Neural Networks Without a PhD: Components of a Neural Network

Sun, 26 Mar 2017 07:38:13 -0400

Neural networks (also referred to as connectionist systems) are a computational approach, which is based on a large collection of neural units (AKA artificial neurons), loosely modelling the way a biological brain solves problems with large clusters of biological neurons connected by axons. – Wikipedia

In essence Neural networks are digital representations of their biological counterparts, and while that sounds intimidating – unless you have a strong background in statistics or cognitive science – NN are not that complicated once you understand the individual building blocks for a Neural Net.

For illustration purposes let’s take a look at the simplest possible Neural Network, the perceptron:

The perceptron is a computational model of a single neuron, and as we can see it consists of 3 basic neural nodes, each of the with a unique function:

Inputs: Also referred to as sensors are in charge of communicating with either software or hardware and passing signals(data) to the Neural network, for example data sources in the case of software or a webcam in the case of hardware.
Neurons: A basic unit of computation, it takes input from other nodes(sensors) and computes and output.
Output: Also referred to as actuators, allow our NN to interact with its environment, outputs can be hardware or software, for example return a true or false value or activating a stepper motor.

Layers

Each node type will be typically arranged in a single layer, with the exception of Neurons which can have many more layers, to clarify let’s take a look at a more complex Neural Network type, a Feed-Forward Neural Network:

The input and output layers are self-explanatory, but what is a hidden layer?

Hidden Layers

The hidden layer(s) is the collection of one or more layers of artificial neurons that are in charge of doing the computation, transforming the inputs into something the output layer can use. In order to understand what happens inside this hidden layer, we need to first understand the concept of emergence:

In philosophy, systems theory, science, and art, emergence is a phenomenon whereby larger entities arise through interactions among smaller or simpler entities such that the larger entities exhibit properties the smaller/simpler entities do not exhibit. – Wikipedia

Each hidden layer neuron has an internal mechanism that governs if the neuron becomes active and sends a signal of its own to the next layer – which can be another hidden layer or the output layer itself.

If we want to break it down, we can then say that a Neural network takes one or more inputs, performs some sort of computation and returns a value – often in the form a vector – that can be used for further processing.

How Neurons Make Decisions

In order to understand how neurons compute the input values and in turn make decisions – aka return an output – we need to add two more elements to our neural network understanding.

Connections: The individual connections between each node (inputs, neurons, outputs).
Connection Weights: Each individual weight represents the strength of the connections between nodes. Another way of thinking about it is how much each ‘neuron’ cares about a particular input value.

With those two new components let’s review our original perceptron diagram:

Alright, now that we have expanded our knowledge about how neurons are connected let’s take a look at how they make a decision based on their inputs.

The Activation Function

Each neuron will take all the connected inputs with their corresponding weights and apply what is called an activation function and return a single output value.

Now, here is where we could start going into the mathematics behind neural networks; but as promise but for now we wont; all we need to know is that there are a number of common activation functions that can be used with Neural networks. To mention a few:

Step Function
Sigmoid
Linear
Gaussian

In the case of our perceptron will be using a step function, will return one of two values 0 (OFF) or 1 (ON), this means our neuron will always a binary value – that is, the neuron is either firing or not, and the graph representation of that function looks something like this:

Putting it all together

Let’s quickly put all we learned together and write a simple perceptron:

This python perceptron is doing a couple of things we haven’t covered yet, like training, and adjusting the connection weights based on the error. For now, the only important thing to understand is that our perceptron takes 3 inputs, applies a step function to such values and attempts to predict the output value.

Running the perceptron will return something like this:

Training Perceptron for 1000 iterations Starting weights: [ 0.90185083 0.75965753 0.10658775] ....................................................... Training completed Weights after training: [ 0.90185083 0.35965753 -0.49341225] Running trained Network against Test Data [0 0 0]: 0.0 -> 0 [0 1 0]: 0.359657525652 -> 1 [1 1 1]: 0.768096100245 -> 1 [0 0 1]: -0.49341225179 -> 0 [1 0 0]: 0.901850826383 -> 1

The result

Can you spot the pattern that the Neural Network is supposed to predict? Leave a comment if you think you figure out the pattern and how well our perceptron did.

Summary

We covered a lot of content in this second post of the series and we now have a much better understanding of the individual elements in neural networks, next we are going to cover ANN topology or how to put organize and connect all of our Neurons.

Neural Networks Without a PhD: Introduction

Sun, 26 Mar 2017 07:38:09 -0400

There is arguably nothing more exciting happening right now in software development than all the new research, and advances in Artificial Intelligence, terms like Machine learning, Deep Learning and of course Artificial Neural Networks have become more common in the developers vernacular.

These are exciting times for AI and not without reasons, as the technology continues to advance at an unprecedented pace and becoming more embedded on our daily lives, with more novel applications reaching the news on almost a weekly basis.

In the last year, I got hooked into machine learning and AI; and I’m firm believer that AI will drastically impact how our relationship with technology in almost every single market and niche you can imagine.

One particular aspect of AI that has caught my attention is the development and application of Artificial Neural Networks or ANN. Personally, I find the concept of emulating the organic brain processes not only interesting but also quite promising; on top of that ANN is one of the technologies that has seen a lot of progress in the last few years.

However, we are nowhere near to reaching the point of democratization of this technology, that is to say that the barrier of entry is still fairly high – albeit consistently dropping. For many developers, and enthusiast this means that many aspects of AI are still far from reach.

If you are like me, you are a software developer with potentially a Computer Science background or maybe a self thought developer with years of experience as a professional developer; and yet when looking at AI programming like Neural Networks, you feel fairly out of your depth and with what seems an insurmountable knowledge gap ahead of you.

This Series

The objective of this series is twofold, first: Become a sort of learning notebook for my own learning into the basic and essential knowledges required to work with artificial neural networks and second: make my notes available and structure in a way that are available for any other developers that are faced the same challenges as I currently do.

The topics I have selected to focus on are the following:

Part 1: Introduction
Part 2: Components of a Neural Network
Part 3: Topologies
Part 4: Supervised Learning
Part 5: Unsupervised Learning
Part 6: Gradient Descent
Part 7: Back-propagation
Part 8: Building a Simple Neural Net
Part 9: Adding Some Bias

Keep in mind that this series is by no means a comprehensive breakdown of each of the topics mentioned above, rather enough of a breakdown in a way that is useful and allows me and you to apply and work with said concepts when working with Neural Networks, so don’t expect a deep dive in math or theory; rather I’m keeping math and theory as light as possible – within reason.

Finally, as I approach this series from the perspective of a student and without the formal academic background there is plenty of room for inaccuracies and errors, so if you find one don’t hesitate to leave a comment or contact me directly.

Happy Learning!!

2017: Simplifying and Regaining Focus

Fri, 06 Jan 2017 05:14:40 -0500

Another year done and over, and as is tradition there needs to be a new year-end blog post, clarifying how I’m going to start doing everything right on 2017 and be a 10x human being, yes 2017 is the year to optimize, simplify, achieve all our goals and wildest dreams …. except all those blogpost are frankly bullshit.

Personally, 2016 was a rough year full of stress, problems and an overall feeling of dissatisfaction; I got close to fully burning out more than a few times.

Now, not all was negative or doom and gloom plenty of good things happened on 2016; looking at the big picture 2016 doesn’t feel like a loss but it doesn’t feel like a win either.

For many a change in the calendar represents an opportunity to change, simplify and fix what they perceive wrong with their jobs, relationship and overall in their lifes.

However one doesn’t need to wait for a random calendar date in order to make meaninful change, change can start at any moment.

Finally, 2017 for me is the year to slowdown and simplify; the biggest mistake that I made on 2016 was trying to do way to much and as result I ended up spread too thin, so I’m slowly working only a few things at the time before moving to the next project.

To all of you, make 2017 about what matters to you; slowdown and focus and really work on the things you conside important, chase your passion and above all take care of yourself.

Machine Learning: A Simple Neural Network

Mon, 06 Jun 2016 08:31:46 -0400

For the last few months I’ve been slowly getting my head around the concepts of machine learning, artificial intelligence and neural networks. The impact that machine learning could potentially have on everyday tasks and the way we develop and interact with applications is incredibly exciting; for that reason I been slowly trying to spend as much of my free time as possible trying to learn this big and complex field.

As part of that learning process I been get a better understanding of what neural networks are, how they work and what kind of applications they can be given, in this first blog post I’ll start by creating a simple Neural network and in further post build up on this simple neural network.

What are neural networks?

Artificial Neural Networks(ANN) or simple referred as neural networks are at it simplest definition is a computational systems conformed of highly interconnected components which capture and represent complex input-output relationships. In essence ANN are a rough attempt to emulate the functionality of biological neural networks.

Typically neural networks are comprised by multiple layers of “neurons” or nodes, as well there are many variations and combinations of ANN like Convolutional Neural Networks(CNN) and Recurrent Neural Networks(RNN); in our case we are going to create the simplest example possible, a single layer, single neuron Neural network.

A simple Neural Network

Let’s start by defining the problem our network will be trained to solve:

We are trying to find the value of the last example, know for anyone paying enough attention, the output value will match the value of the input left-most column; let’s see if we can make our neural network answer the question correctly.

Based on the examples inputs and outputs our neuron will need to take 3 inputs and return a single output.

How does Training work?

We have a training dataset, so now is time to train our neuron based on the known examples, but how does this process work? are we really teaching a computer to think?

Well no, not really the process could be more accurately described as an educated guess by the computer. In order to train neuron we are going use a technique called backpropagation, which according to Wikipedia is:

Backpropagation, an abbreviation for “backward propagation of errors”, is a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent. The method calculates the gradient of a loss function with respect to all the weights in the network. – Wikipedia

Clear right?, yeah no worries I got stumped as this point as well; let’s dig a little into another concept of our neural network in order to understand what’s happening during the training a little better.

As we mentioned before our neuron will take three inputs per example, each of this inputs is connected to our neuron by a “synapse” the way training works is by assigning each input synapse a weight that can go from a positive to a negative number.

In our case we will be training against a know dataset, so the process will roughly look as follows:

Load the training set from the known examples.
Set random weights for each input.
Pass the input and the weight to a mathematical formula to calculate the output.
Calculate the error, that is to say the different between the neuron’s output and the known value.
Based on the error adjust the weights slightly.
Rinse and repeat, 10,000 times

By doing this will eventually come to an optimal set of weights for our training set; assuming that our unknown examples follow the same patter then the neuron should be able to make an accurate prediction. This in a nutshell is what we call backpropagation.

The Math

Now, you might be wondering how the actual output values are calculated; in this case we are using a Sigmoid function to calculate the output values. Let’s quickly take a look at the math involved.

A sigmoid function is a bounded differentiable real function that is defined for all real input values and has a positive derivative at each point. – Wikipedia

I don’t want to go to deep in the math of how and why this kind of logistic function is better suited for what we are currently trying to do, for now its enough to know that a sigmoid function will work for use case and it has an easy derivative to calculate.

The plotted result of the above function will look like the following curve:

Keep in mind that this is not the only type of neuron, and depending on the kind of neural network and job we are giving it we might use other kinds of functions to calculated the output.

Adjusting the weights

The second part to this neuron code is to be able to adjust the synaptic weights during training, in order to do that we need to provide a second function that will calculate the adjustment.

In this case we will make use of another machine learning formula “Error Weighted Derivative” or delta rule.

Or better represent by:

This formula will adjust the weight proportionally to the output of neuron; by using the original sigmoid curve as part of the calculation, we make sure that the more confident the neuron is the smaller the adjustment will be.

Show me the code

Ok enough math and theory, let’s go ahead and write some python. First and foremost we will need to include the following methods from the numpy library:

array: used to create matrices
random: generates a random number
exp: calculates the natural exponential
dot: used to multiply matrices

Next we will need to define the training set inputs and outputs, for that we can split them into 2 separate matrices:

Following that we will need a variable to hold the unknown input:

As well we need to generate the set of weights for each one of our inputs:

The code above sets the seed for the random number generator, doing so will return the same random numbers between runs. Let’s print the synapse_weights to see the results:

Which should generated something like the following:

[[-0.16595599] [ 0.44064899] [-0.99977125]] It doesn’t matter how many times we run our example our initial weights will always be the same.

Now, what would happen if we where to run our neural network without training? What do you think would happen? Would be get an accurate result?

Let’s find out:

Let’s look at the last line we just added, take a few seconds to read the code. All that function is doing is translating our sigmoid function into code, which will always return a value between -1 or 1.

Go ahead and run the code above, you can expect to see something like the following output:

[ 0.45860596]

And as you can see that value is not terrible accurate, I guess we kinda, coulda, maybe say the value is close to 0 but that won’t do it, we are looking for a value as close as possible to either 1 or 0.

With that said the only remaining step is to train our ANN before actually calculating the output, and we will do that by running the sigmoid function 10,000 times and tweaking the weights on each iteration.

The code above will let us take a peak at the trained synapse weights, which look like the following in my case:

``` # Before training [[-0.16595599] [ 0.44064899] [-0.99977125]]

After training

[[ 9.67299303] [-0.2078435 ] [-4.62963669]] ```

That’s some difference, but will that result in any improvement? Let’s find out by running the following and final snippet:

Resulting in;

[ 0.99993704]

Bam! I would said that is a pretty accurate result. Experiment tweaking some of the values or trying to calculate a different example.

Conclusion

You made it to the end of the post!! Congratulations! This one was a long one and by no means a necessarily easy one to follow. However, keep in mind we are just scratching the surface, I skimmed on a lot of concepts and theory, the math alone could take several posts to start from scratch.

References

Functional Programming: The Paradigm for the Next Generation

Mon, 02 May 2016 13:05:11 -0400

Functional programming is often treated as the fad of hipster mustachioed programmers, and more often is dismissed without much consideration; after all Object Oriented Programming is superior. Well, at least that use to be the trend but in recent years Functional programming has experienced a renaissance and as result functional programming languages are gaining more and more adoption.

But although functional programming is becoming more popular there is still a lot of friction and resistance to change specially when it comes to well seasoned OOP developers that are trying to make the transition, probably one of the best examples is the article titled Functional Programming Is Not Popular Because It Is Weird. which starts with the following paragraph:

Writing functional code is often backwards and can feel more like solving puzzles than like explaining a process to the computer. In functional languages I often know what I want to say, but it feels like I have to solve a puzzle in order to express it to the language. – Malte Skarupke

As programmers we are trained to conceptualize and abstract problems in a certain way, our first instinct is to try and frame the problem using objects a concept all to familiar both inside and outside of programming.

Functional programming takes a different approach, one that might feel alien, strange and backwards to most programmers; one that forces us to leave state behind and embrace immutability, to favour recursion instead of loops, to leave what we know and adopt new patterns to solve old problems.

For some developers this is a huge change, one of herculean proportions; but I’m happy to say that is not a pointless one, quite the opposite there are many rewards and benefits to discover once you fully switch your mindset to think in a functional way.

Functional Thinking

Functional Thinking? What do we mean by that exactly? Well, I could go into detail about what functional thinking is how writing functional code is more than just the selection of the programming language but also affects how we design our code, how we structure reusable blocks of code, what trade-offs we accept and so on.

However, I believe the simplest way to clarify the concepts of functional programming and thinking is by comparing two solutions to the same problem, one solution using an imperative language and the other one using a functional language.

In this case I’ve selected two languages that I’m familiar with for the imperative language we will be using PHP and on the functional side we will use Elixir.

Let’s start with a simple example, iterating through an array, adding each of the values and returning the sum. Let’s start with the solution in PHP:

Simple enough, by using the foreach loop we iterate across every single item in the array added to the sum integer and return the total, when its done. Let’s take a look at the Elixir equivalent:

Now, your initial reaction might be something like What in Knuts name is that?!, Why are there two declarations of the same function?!. If this is your first time seeing any Elixir code the above snippet might look awfully confusing and hardly making any sense.

But in reality it’s actually very simple code, taking advantage of a common patter in FP and one of Elixir strongest features; pattern matching. Let’s start by breaking down the code:

elixir defmodule SumModule do

Ok, nothing new to see here; this code might look familiar for anyone with previous experience with the Ruby programming which shouldn’t be surprising considering the creator of the Elixir language was a long time contributor to the Ruby language.

Moving to the next two lines:

elixir def sum_list([]), do: 0 def sum_list([head|tail]), do: sum_list(tail) + head

Here is where the magic happens and where developers with no exposure to functional programming will get confused, how is it possible that we have 2 definition for the same function.

In elixir this technique is called pattern matching, and the code would read as follows:

When function sum_list() is called with an empty list, return 0
When function sum_list() is called with a non empty list, grab the first element of the list, call sum_list() with the remaining of the list, finally add the head value to the result

This technique is extremely common in elixir, handle each edge case in a separate function head. No need to worry about complicated control structures.

Now, I could argue the resulting code is cleaner and I dare to say more elegant, but before making statements like that, let’s make our example more interesting, and change the array to the following values:

php $sumList = [5,'four',2,'ten','one',28,6,'five'];

Perfect, so now our code has to check if the value is a string or an integer and then convert the string representation into an integer for addition. To make things a little easier we are going add another array with integer values so we don’t have to worry about the conversion.

Well that does the trick but the code is definitively getting more complex and is no longer that easy to follow; let’s take the same problem and solve it using elixir:

In both cases the results are the same however, with elixir and pattern matching we get more compact code that doesn’t rely on state, and instead used recursion with strong, well defined conditions. While the examples might not look overly complex and I’m sure they don’t illustrate each language strongest features; they do serve the purpose of illustrating the difference in thinking methodologies and how each the paradigms approach a specific problem.

Conclusion

In my opinion the benefits of functional programming are very real and valuable if only for the fact that introducing new paradigms to your problem solving toolbox can make you a better programmer; even if in the end you continue using an imperative language.

Back in the Saddle Again

Sun, 24 Apr 2016 11:55:50 -0400

It has been over 4 months since I posted anything online, in the meantime the world has kept spinning and there are many updates, changes and news to talk about; so why did I suddenly decided to go silent? Well there are a myriad of reasons but primarily it was due to my change in role at Demac Media, for the last 5 months I been operating as the Director of Engineering.

The title might sound fancy but in essence is the same responsibilities that I been executing officially and unofficially, which include taking care of the developers training and education, code quality, server and infrastructure and developer tools and methodologies.

So far is a role that I enjoy immensely but that said it’s not without its challenges, more than anything is the scope and size of some of the tasks and how time consuming they can be. Time, there never seems to be enough, right?

The regrettable side effect of this transition was felt by my personal projects, my writing and overall hobbies and activities outside office related work. Thankfully things have started to stabilize and I’ve gotten better at managing my time and priorities; and with that it only felt right to get back into the saddle and start putting some time and energy back into this blog and other related side projects.

I’m glad and really looking forward to putting some useful content out and now specially that there are so many new topics to talk about from Magento2 to my new obsession Machine Learning and AI.

On Codes on Conduct

Sun, 24 Jan 2016 19:41:40 -0500

Recently, there has been a lot of controversy and discussion surrounding the adoption of a code of conduct for the PHP project; there has been endless back and forth between proponents and opponents of the motion. I’ve debated if I should publish my opinions on the subject or simply remain quite and hope for the best; however as emails from the internals list continued to flood my inbox, I found it harder and harder to just sit idly silent.

The following article(s) are heavily opinionated and there is a very good chance that you will disagree with my position and my arguments, the topic by is very nature is extremely contentious and of a sensitive nature. However, they say that disagreement fosters communication and the exchange of ideas; and I’m hoping that is true in this case. With that said, let’s jump into the topic at hand and start talking about code of conduct.

Codes of Conduct

A code of conduct is a set of rules outlining the social norms and rules and responsibilities of, or proper practices for, an individual, party or organization. Related concepts include ethical, honour, moral codes and religious laws. – wikipedia

Albeit general the definition above does a good job of describing the main goal of a code of conduct. Simply speaking, codes of conduct dictate the way individuals should act and communicate with each other in what is deemed a relevant space for a project. Now, inherently there is nothing wrong with the concept of code of conduct as we all can agree that we should strive to provide a welcoming and respectful space.

Unfortunately, as most things in the real world it is not as simple as it sounds and we need to at least be aware of the consequences, side effects, and political baggage documents like a code of conduct can have.

Let’s start by taking a closer look to the Code of Conduct original proposed for the PHP project:

That document is called the contributors covenant, a template for the code of conduct currently in use by around 10,000 open source projects — impressive number, specially considering some of them are quite large. Before I jump into some of my more controversial opinions about this document, let’s analyze the language and the document itself solely based on its content.

Let’s break it down paragraph by paragraph.

As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.

Ok so far so good, there is nothing wrong or arguable with the first paragraph; striving to make our community more open and welcoming is something all of us can agree with.

We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality.

Here the waters start to get a little bit muddy, while at first glance everything seems reasonable enough, by taking a look at the language we can see a couple of issues; those mainly being that the list of harassment topics is not wide enough as well as the fact that the paragraph doesn’t set the scope on which the project maintainers are supposed to act on.

Think about for a second, under the current covenant any comment, retweet, blog post, etc., could be taken in consideration for a harassment or discrimination complain; even if it has nothing to do with the PHP project.

Let me present a better version of the second paragraph:

We are committed to evaluating contributions within project channels (such as reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other project activities) without regard to the contributor’s level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, politics, or activity outside of project channels.

On it we limit the scope to only the project channels — which should be clearly defined in a later section — as well it accounts for political opinions and views of the contributors and defends them from discrimination. Let’s move on to the next paragraph:

Examples of unacceptable behaviour by participants include:

- The use of sexualized language or imagery
- Personal attacks
- Trolling or insulting/derogatory comments
- Public or private harassment
- Publishing other's private information, such as physical or electronic addresses, without explicit permission
- Other unethical or unprofessional conduct

Another paragraph that at first glance is hard to disagree with, but reading carefully we need to define what is considered personal attacks, trolling or unprofessional conduct.

Again open and vague language can lend itself to abuse, specially on a document that calls for drastic punitive action like this one does.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviours that they deem inappropriate, threatening, offensive, or harmful.

And now we get to the part that causes me the most concern, the paragraph above describe the list of sanctions and punitive action that must be taken for anyone that is found violating the CoC.

Regardless if is temporary or permanent the act of banning someone from the project can have real world consequences and affect people’s livelihood. I’ll clarify what I mean with real world consequences and why I believe that whole idea of punitive actions is highly misguided further down the article.

By adopting this Code of Conduct, project maintainers commit themselves to fairly and consistently applying these principles to every aspect of managing this project. Project maintainers who do not follow or enforce the Code of Conduct may be permanently removed from the project team.

I kept re-reading this paragraph, and it keep bothering; as a project maintainer you are not only bound to make the project fair and welcoming but you are bound to follow the covenant to the letter or get permanently banned yourself.

Again, the emphasis is in the punitive action; follow our rules or else!

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community.

Finally, a mention of scope and limits … but no wait that’s it ? Correct, again extremely open ended and vague language; you could argue that you are always representing the language in public and private, there are no boundaries; not for your own point of views, nor for personal life.

Vagueness can lead to situations where people get easily banned for comments, views or postings that are completely unrelated to the actual project; it doesn’t matter if you say that your tweets are ‘performance art’; if someone deems them offensive you run the risk of getting labeled and thrown away from the project.

This exposes another weakness or oversight in the contributors covenant code of conduct, it doesn’t account for intent, nor it takes provisions to restrict who can and can’t report issues. Brandon Savage brought this point masterfully on the PHP mailing list — you can read the full email here—.

I won’t cover the full extend of the issue as presented by Brandon, but here are his proposed inclusions and corrections to the CoC:

The Code of Conduct should specifically state that a person who is not a direct party to the alleged incident is not permitted to make a complaint.
We should require that any person who is accused of violating the Code of Conduct clearly have intent to do so. This is a harder standard to prove, but one that should help us from having to deal with edge cases. A death threat is a clear-cut case of intent, for example.
The Code of Conduct should be modified so that abiding or not abiding by it is demonstrable with evidence, taking “feelings” out of it entirely. For example, a person shouldn’t be in violation of the code because someone “feels harassed/trolled/etc”, it should be because they’re ACTUALLY harassed/trolled/etc.
The Code of Conduct should bar filing a claim of harassment if harassment from both parties towards one another can be demonstrated. This avoids a race to the courthouse by one side to punish the other in an argument.

All worthy additions to the code of conduct to prevent abuse and limit the number of edge cases and people affected by them.

Punitive action and the real world

Many of the concerns that have been voiced regarding this CoC are related to the apparent focus on punitive action; this CoC is at its core a laundry list of forbidden behaviour that people are required to avoid and to ensure compliance the threat of the perma-ban hammer hangs over their heads.

Unfortunately, this will likely not end with just one person getting banned from the PHP project but it could potentially end up having more serious repercussions. So far we have discussed the CoC in the PHP context; but let’s remember that the current CoC is based and following the Contributors Covenant, which has the issues I listed above.

Currently, over 10,000 projects follow this contributors covenant as it’s currently provisioned and enforced, the following scenario is not outside of the realm of possibility:

Once a developer is deemed in violation of the Contributors Covenant what stops any of the other covenant projects from saying ‘Well he/her was labelled as an harasser/bigot/sexist by X project why should we allow him to do the same thing here in project Y less ban him/her before they can cause any harm’.

Now, before I get accused of fabricating straw-man arguments or exaggerating let me clarify there is no indication or record that the scenario has occurred nor that there are any plans to do something of that nature.

Recently, I listened to a prominent figure inside the PHP community talk about the subject, all I can say is it was disappointing to hear said person take such a dismissive attitude towards the concerns of people that oppose to the current CoC.

If we are truly talking about creating a welcoming and open community, then we need to listen and address the concerns of detractors as much as we do for the proponents of the CoC; after all, the goal is equality and fairness; is it not?

Even without accounting for this scenario, getting banned from contribution under the accusation of misconduct could cause problems in the real world and their work; you could get you banned from conferences, affect publishing deals, in short, it could royally mess up with your career.

Next steps forward

As it stands, the current proposal for the Code of Conduct has hit an impasse; the more the discussion seems to go on, the more it seems that the proposal is attempting to setup a mini-judicial system; and speaking candidly, the PHP community is not capable of pulling this off.

There are so many variables and processes, not to mention, the balances and checks that there would need to be in place in order for this to be successful. For that reason, I would like to put forward my own version of a code of conduct that I feel would be a better fit for the PHP Community, if anything at all, take it as a source of ideas coming from a different perspective.

I call it The Pragmatist Code of Conduct

Exploring Elixir Recursion and Lists

Sun, 17 Jan 2016 15:46:54 -0500

Recently, I’ve been playing with a new language called Elixir. Elixir is a functional programming language specifically designed with the intention of creating scalable and maintainable applications.

As many developers my main experience with programming languages and paradigms has been with object oriented programming; functional programming is a completely different beast and as I explore and learn more, the more fascinating I find the promises and ideas behind functional programming.

One of the things that has been messing with my head considerably is the fact that functional programming relies heavily on recursion. Now, technically speaking elixir is not a purely functionaly language and the way they implement immutability is a bit different; that being said in elxir as in any other functional programming language there are no loops.

No loops!? no foreach, no for, no while instead we get recursion; where a function gets called recursively until a condition is reached that stops the recursive action from ocurring again — for example processing each element in a list until the list is empty.

Sounds confusing right? Let’s look at an example:

The previous code is part of an exercise I did for Exercism.io that required to reimplement the functionality of the native List module; the count function will take a list of numbers and return a count of the elements inside the list.

If this is the first time you look at elixir the first thing to jump at you is that it seems we are defining the count function twice; this is not a mistake and is one of the most amazing features of the elixir language, it’s called pattern matching I won’t into detail on how that works on this post but suffice to say that the first definition will only be called if we recieve an empty list.

Back to recursion, let’s take a look at the second defintion of the function:

elixir def count([_|tail]) do count(tail) + 1 end

All we are doing at this point is taking out the first element of the list passing it to the count function and returning the result + 1 on every single instance, but how does that work exactly let’s break it down assuming we pass [1,2,3,4,5] as parameter:

elixir count([1,2,3,4,5]) #returns 4 + 1 "this in turn calls" -> count([2,3,4,5]) #returns 3 + 1 "this in turn calls" -> count([3,4,5]) #returns 2 + 1 "this in turn calls" -> count([4,5]) #returns 1 + 1 "this in turn calls" -> count([5]) #returns 0 + 1 "this in turn calls" -> count([]) #returns 0

This is a very simple example of the power of recursion but illustrates the concept very well. For now the most imporant thing about recursion is that we don’t need mutable state while iterating through a list of values — a.k.a solving a problem.

2016 the Year of Quantified Self

Sun, 03 Jan 2016 11:10:06 -0500

Quantified self and self-improvement have been on my radar for a few years now, and with a new year starting I finally decided to give it a serious try and see what outcomes I get from it.

For anyone not familiar with the concept, quantified self is a movement to incorporate data acquisition technologies on aspects of a person’s daily life; for example mood over time, food consumed, weight, heart rate, etc.

You Can't Improve What You Don't Measure

Quantified-self focus on tracking three principal categories:

Inputs: Food consumed, water intake, etc
States: Mood, heart rate, blood oxygen levels, stress levels
Performance: Mental and physical

Self-monitoring is already more common than what people realized, just take a look at their wrists chances are you or someone you know uses a fitness tracker already.

Fitness trackers like the Fitbit or Up are what I would call entry level self-monitoring devices; since they are often limited to simple fitness tracking like steps walked, calories burned and in some case basic heart rate.

However, in term of quantified self, data is king and the more of it we have, the better, right? Well, that is true to some degree —we can always drown in meaningless data— when thinking about what kind of data we should collect we need first to consider the outcomes. That is to say what information can I get from the data collected? Will it give me actionable items?

More often than not, a single metric won’t give you the full picture when you look at it in isolation, but when look in correlation with another data is where we might start learning interesting things about ourselves, for example, a correlation between poor sleep and the number of coffee consumed on a given day, or water consumption vs. daily mood.

This is to me the fascinating part of self-monitoring; it’s not enough to have healthy habits if we don’t understand how those habits affect the big picture and our overall enjoyment of life.

In that sense I’m starting 2016 with self-monitoring as one of my top priorities; trying to track as much as possible but making it as easy and low effort as possible; automating where I can and reducing friction where possible.

I’ll go into more detail on the tools and software that I’m planning to use in a future post; as I’m still playing and tweaking my setup.

On New Year's Resolutions

Fri, 01 Jan 2016 10:41:30 -0500

Another year came and went, and as we do every year we are expected setup new resolutions and new goals for the upcoming year. It’s the usual thing to do that society expects of every single one of us.

I can almost hear family and coworkers asking — So, Any new year resolutions?— And like every year I’m tempted to answer yes and list some of the many things on my ever-growing todo list:

Learn guitar
Finish that book I been working on for months
Do more exercises
etc. etc. etc

But this year I’m done deluding myself and setting up goals that I know I won’t follow, my only resolution for 2016 is simple:

Be Happy

Being happy means I will spend more time on the things that bring me joy and happiness and less worrying and suffering on doing things I don’t like; it’s as simple as that, do more of what I like and less of what I dislike.

On 2016 what will you do to be happy?

TDD is not Dead

Thu, 03 Sep 2015 00:00:00 -0400

So, where does this whole TDD is DEAD thing came from? Well, it all started with let’s say a provocative talk and follow up blog post by David Heinemeier Hansson (@DHH) where he expressed his frustration with testing and put into question the value of TDD.

To break down DHH’s points and concerns against TDD:

Developers make you feel like your code is dirty if you don’t practice TDD.
Driving design from unit tests is not a good idea.
TDD notion of “fast tests” is shortsighted.
100% coverage is silly.
TDD created test-induced design damage.

TDD is not dead, not really. And it won’t really ever be dead, it will change or be replaced with something better; in fact it already has, and in my Magento Extension Test Driven Development book we focus on Behavior Driven Development, an approach that emerged from the original TDD methodology.

Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development. – Wikipedia

I’ll quickly address each of DHH’s points:

1.- Developers make you feel like your is dirty if you don’t practice TDD

I agree, this sucks! Developer shaming is never a good thing. Rather than shaming a developer that doesn’t do TDD or testing; teach and share. That’s the best way to help the community grow.

2.- Driving design from unit tests is not a good idea

I don’t agree. Tools like PHPSpec do an incredible job helping us not only to test our code, but to drive our design and produce cleaner code.

3.- TDD notion of “fast tests” is shortsighted

Well, not true at all. When working on a TDD cycle, you want your test to run as fast as possible (within reason) to get that constant feedback and validate your code.

4.- 100% coverage is silly.

I have to agree on this one. Be pragmatic about your testing. Test only when it matters and think twice to see if your test is adding value.

5.- TDD created test-induced design damage.

Again, I have to disagree. Properly used TDD/BDD can lead to clean, maintainable, and easy to understand designs. Blaming TDD for poor design is like blaming your IDE for poor code indentation; be pragmatic with your tools.

Testing sounds like too much work / Testing will make my development slower

TDD does add a bit more overhead, specially while you are still getting familiar with the tools and the process. So I won’t lie to you, chances are that your process will get slower in the beginning, but as you get better with the tools and more accustomed to the testing process, you’ll become faster.

Think about writing tests as an investment account; you will see a greater return of your investment with time.

Also with Phpspec, our tests are thorough code examples and a form of living documentation. But how does all that apply to Magento?

Magento 1.x is not testable, it is impossible to write tests for it.

Testing Magento 1.x has always been a difficult topic; Magento (at least in its current iteration) doesn’t lend itself too well to testing.

Over the years, there have been several attempts at providing some degree of testing to the platform: from the official MTAF (Magento Testing Automation Framework) to the community-driven tools like Ecomdev’s PHPUnit extension.

For the longest time, testing was something that wasn’t discussed that much in the Magento community, and honestly it is not surprising, since building proper tests for Magento extensions can be difficult even for seasoned veterans; let’s not say the junior developer just getting started with Magento.

Magento 2 and the Grail of Testing

Magento 2 is changing that for the better by using design patterns like dependency injection and by having a massive test suite from the start. As with any change, you can see some resistance and skepticism among the community members. To be honest, it’s not without reason, I for one find the concept of 100% code coverage misleading. After all, useless tests do exist. With all that being said, I’m hopeful of the changes that Magento2 is introducing both to codebase and the developer community. I truly believe that for the most part they are positive ones.

At the end of the day, it is the community’s opinion what matters, so what do you think? Is Magento2 overusing dependency injection? Is the test suite useful? How will the introduction of all these new paradigms change the way you develop Magento extensions? Don’t hesitate to leave a comment below.

P.D: Parts of this post were taken from my new book Magento Test Driven Extension Development if you are interested to learn more about TDD/BDD development in Magento go ahead and click the link!

Manfred Macx and the Problem of Information Overload

Fri, 17 Jul 2015 00:00:00 -0400

I’ve always been fascinated with the technology concepts presented on Charles Stross book Accelerando, artificial cats, downloads of consciousness, nanotechnology, etc. but of all of them one got my particular attention; Manfred Marcx glasses.

Glasses? What’s so special about the glasses? Well this are not an ordinary pair of glasses, they are in fact augmented reality glasses, informing Manfred about recent news, results from search agents, phone calls, navigation instructions and what ever information he desires.

We could call them smart glasses, in the book this glasses allow Manfred to be on the bleeding age of technology and information; to the extreme without them he is incapable of functioning.

In Charles Stross’ Accelerando, the arrival of the technological singularity is prefaced in the first section of the novel with the story of Manfred Macx, a hyper-connected, post-economic entrepreneur who relies on reputation and good-will rather than traditional monetary wealth. – Source: Post-human, post-memory: Mnemotechnology in Charles Stross’ Accelerando

The attractive part here is not the augment reality glasses themselves but system that powers them, an exocortex.

An EXOcortex can best be described as the portion of a trans- or posthuman entity’s brain (or cortex) which exists outside of that entity’s primary computing structure, usually the brain inhabiting a person’s “meatbody.” For example, a person’s exocortex could very well be composed of all the external memory modules, processor, and devices that the person’s biological brain interacts with on a realtime basis, thereby in effect making those external devices a functional part of the individual’s “mind.” – Source: Wikibooks

While we are still years if not decades away of being able to directly enhance our biological brain with help of external hardware, current technology should be more than sufficient for creating at least an early version of the software an exocortex system would need.

Defining the implementation

The first instinct for a project like this would be to go and develop a complex architecture, maybe a multi-agent system; yes that would be cool …. and also extremely complicated. But, what if switch the concept a little bit, what if instead of having a swarm of intelligent agents, we use a micro-service architecture.

Could that work? Will that architecture work or fail miserably due to:

network latency
message formats
fault tolerance
deployment complexity

NOTE: The system should not only help with data consumption and processing but also with information generation.

If we’re not doing something with the information we’re taking in, then we’re just pigs at the media trough.

Here is where the challenge comes in, I can build a system that feeds me information, sure. But can I build a system that, helps me to generate content faster and more efficiently.

The metacortex is the Web — our growing global network of information, people, sensors, and computing devices.

Also would that be considered an exocortex or more akin to a metacortex, all relevant questions but besides the point of an initial prototype implementation.

More interesting are the advantages of a distributed system like this, and the mailability it could offer. A system that is comprised of microservices wouldn’t care about the implementation of each service; as long as they respect the common contract for communication.

The Web is literally a new layer of the human brain that transcends any individual brain. It is a global brain that connects all our brains together. It is intelligent. It is perhaps humanity’s greatest invention. It collectively senses, reacts, interprets, learns, thinks, and acts in ways that we as individuals can barely comprehend or predict, and this activity comprises an emerging global mind. – Source: Nova Spivak

Although even in a distributed service architecture there has to be a central point of orchestration, a system or process that is in charge of defining and running the tasks.

Option 1

Create a chain of events and services
Each service in the chain calls back to the next service in line
The task runner is only concerned about starting the chain of events

Option 2

Decoupled services
Central system implements adapter for each
Runs each step of the task passing the relevant data
Can track the progress of each task.

At this point in time Option 2 seems more attractive, it also gives me the flexibility to integrate third party services into this structure without having to code the service myself. And I can leverage sites like mashape.com to help me with at least the initial implementation.

An exocortex is a theoretical artificial external information processing system that would augment a brain’s biological high-level cognitive processes. An individual’s exocortex would be composed of external memory modules, processors, IO devices and software systems that would interact with, and augment, a person’s biological brain. Typically this interaction is described as being conducted through a direct brain-computer interface, making these extensions functionally part of the individual’s mind. – Source: Wikipedia

Narrowing the Scope

To measure progress and success, I need to define achievable goals first, in order to do that I have to narrow the scope and define what I want to gain from my “exocortex”.

Do I want …

Better memory and indexation
Help with data consumption
Improve my content output
Automated tedious tasks

Well, yes to all the above; but which one(s) would help me the most right now? Based on my current goals, and the current challenges I’m facing; data consumption and subsequently content output would be extremely handy.

Still, the scope seems to wide; we can narrow it further if we set an example use case.

Example 1

EVERY [30min] GET [rss_feeds] OF [www.joelonsoftware.com,blog.cleancoder.com] AND GET [subreddit] OF [programming,coding,php] AND GET [hnews] THEN DO [categorization] of {title} WHEN {categories} HAVE_ONE_OF [tdd,bdd,agile] POST_TO [Summarizer Queue]

Example 2

EVERY [30 minutes] GET [RSS feeds] OF [www.joelonsoftware.com, blog.cleancoder.com] AND GET [Subreddit feeds] OF [programming, coding, php] AND GET [HNews New] THEN DO [Categorization] OF {title} DO [Classification] OF {title} WHEN {score} GREATER THAN > [0.7] POST TO [Direct Email] WHEN {categories} HAVE ONE OF [tdd,bdd,agile] POST TO [Summarizer Queue]

What’s next?

Sounds good so far, I been working on the architecture for this project for the last year or so, and already have a working prototype for data collection. Here is where the fun part begins since I can collect reasonable amounts of data what should I do with them?

I’ll go over the details of the data collection implementation on the next post.

On Encrypting Magento Extensions

Thu, 09 Jul 2015 00:00:00 -0400

The opinions expressed here represent my own and not those of my employer.

Recently there has been much debate over if Magento Connect should allow encrypted extensions and if so in what shape and form. The following post is my honest, uncensored opinion about the whole situation, please read the whole thing you might be surprised by my conclusions.

Its Rant time

Let me start by saying that I absolutely detest encrypted extensions and find IonCube encrypted code abhorrent. Let’s leave aside the philosophical argument about open source and openness.

My reasons for hating obfuscated/encrypted code are much more practical and mundane, it all comes down to having control over the code is running on a specific project, having the flexibility to change it, trusting the code follows certain standards.

Control

As developer any third party extension paid or not that goes into any of my projects has to go through several tools like php-metrics, php-codesniffer, php-messdetector and so on to make sure that the code meets certain standards. Code quality, stability and adherence to standards are things that I consider important and the lack off quality in Connect extensions was my main motivator to create triplecheck.io

Encryption puts code whatever quality it might be on a black box where we cannot check, test or fix it; worse comes to worse we are still responsible for supporting an maintaining the sites running said extension. Moreover, believe me, not every merchant is going to be understanding enough when you tell them

“Well there is nothing I can do to fix the extension, I have to wait for the extension developer to answer back; with luck it will be in a week. Please close your business for the meantime.”

Flexibility

Now don’t get me wrong there are some really good extensions in the market and some exceptional extension providers out there, but that being the case sometimes we need a to add functionality to a third party extension that get us 3/4 of the way to where we need to be.

With an encrypted codebase, that is not an option you get what you paid for and sometimes even less, as a developer I cannot debug or extend a third party extension. I am confined to a little black box of obfuscation, and complete powerless to change it.

Security and Trust

Encrypting code even if is part of the overall extension is asking for merchants and developers to trust on that extension provider, and Big Time. There could be anything hiding inside that encrypted file.

And no, I am not playing with scare tactics here, let put a little code snippet as an example:

public function validate() { return $this; parent::validate(); $currency_code = $this->getOrderData()->getBaseCurrencyCode(); if (!in_array($currency_code,$this->_allowCurrencyCode)) { Mage::throwException(Mage::helper(‘extensionname’)->__(‘Selected currency code (‘.$currency_code.’) is not compatible with extensionname’)); } //mage::log(“Validate “); return $this; }

Yes, this charming piece of code goes for around $300USD the code was taken as it is but the name removed. Good luck trying to figure what was happening if the code was encrypted; extensions like this one are why encryption also becomes a trust problem; can you trust the extension developer not to take shortcuts, follow best practices and not do stupid code like the one above?

Time for a sensible argument

By know I have established my complete dislike of encrypted code, and if that is true the only logical conclusion can be that Magento Connect should not allow encrypted extension on its market place, right?

Well no, after a lot of though and consideration, I do believe that encrypted extensions should be allowed into Connect, and you are probably thinking. Wtf?! after that rant!? you are saying yes to encrypted code!!!

I am and for a very simple reason, Magento Connect should be a platform open to everyone wanting to sell and promote their extensions; is up to the extension developer to decide if they want go that route, and is up to the merchant to decide if they want to deal with the possible drawbacks.

In the end, regardless my or anyone else personal preferences the market will speak. Will encrypted extensions thrive or wither? Is up to the ecosystem and the market to decide.

Git Tips and Tricks

Sat, 04 Apr 2015 00:00:00 -0400

Recently I’ve been collecting a large collection of git shortcuts, tips and tricks. The following commands have been tested on linux and some of them will required a recent version of git.

I expect that many of you will find them useful as much as I do. So let’s get started with some simple but helpful commands to search and review your git repository history.

Show log for only a specific branch

The following command can be used if you are working on a project with a particular dirty history or a log of branches

git log --first-parent {branch_name}

Show the log as a single line

The following command will show the commit has and the first line of the commit comment.

git log --pretty=oneline --abbrev-commit

A more advanced and nicer version of the command above is:

git log --graph --pretty=oneline --abbrev-commit --date=relative

Search for a string in all commits across the entire git history

This can be really useful when looking for a particular piece of code to see when it was added/removed:

git log -S{text_to_search}

Shows branches that are all merged in to your current branch

git branch --merged

And in the same way we can see all the branches that haven’t yet been merged.

git branch --no-merged

Getting Started With Magento and Docker for Development

Thu, 05 Mar 2015 00:00:00 -0500

Docker has taken the DevOps community by storm and is rapidly changing the ecosystem towards distributed architectures, and in case you haven’t heard about Docker here is the quick definition:

Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications.

And while Docker brings amazing things to the table in terms of application architecture, scalability and so on, in this post we are going to use docker to streamline the development pipeline.

The Problem

If you are reading this article, chances are that you work for a Magento Agency and likely your development flow looks something like this:

Developers work on local versions of the site and make frequent changes
Code is pushed to an staging environment for testing
The staging site is tested, and code is reviewed
On Approval, the code is merged into a production branch and deployed to production

Based on the flow above, we can say we have at least three environments:

Local Development
Staging
Production

Now, this is an oversimplified workflow, and as soon you start adding CI systems, UAT servers, etc. things start to get more complicated. For now let’s stick to our three environments.

Ideally local development and staging should be as close as possible to production as we can, in practice, however, more often than not; there will be big differences.

For example, a large chunk of your developers might be using a solution like MAMP or WAMP, each with its own set of quirks, will likely not match your staging/production. This is where problems start happening, if you ever heard the dreaded phrase “It works on my local” then you know what I’m talking about.

In addition keeping staging in parity with production can have a large overhead, especially if you run multiple architectures, or if your sites have different requirements like php-versions or installing something like IonCube.

The Solution

This is what adding docker to our development pipeline is all about, being able to have a consistent environment for development, staging, qa and production. For now I will only talk about the first step on our development pipeline, local development.

Running Docker is very much like running virtual machines for local development but with the following advantages:

Faster creation and initialization
Lower system footprint
Easily shareable between developer and environments

To facilitate things, I’ve created a base docker image that runs a simplified version of server stack I run at Demac Media you can find the link at Docker Hub

The image will setup apache2.4, php-fpm and and the 4 major versions of Php (5.3,5.4,5.5,5.6) as well create vhost macros for easy setup. Let’s take a peak:

This image is not really meant to be used directly but rather as a base for your projects. The way I would recommend using it is as follows:

In your project base create a folder named docker and add the following file:

Filename: Dockerfile

A Dockerfile is a text document that contains all the commands you would normally execute manually in order to build a Docker image. By calling docker build from your terminal, you can have Docker build your image step by step, executing the instructions successively.

All we are doing above is setting up the vhost for our current project, speaking of which we will need to create said vhost file inside our docker/vhost folder, since we are copying it to our newly created image in the following line

COPY config/vhosts/ /etc/apache2/sites-available

So let’s go ahead and create the file

Filename: config/vhosts/localhost.com.conf

Wait, that’s it? Yes, it’s that simple; the example above is using one of the coolest Apache2.4 features, Macros.

The base image has a macro for each of the php versions and as you can see is the first thing we are telling our vhost file to use

Use VHost-PHP5.4

The values after that are variables to be used by our macro file, in this case:

$host
$ip
$port
$dir

And while right now we are not using all of them directly, it opens an interesting set of possibilities. We are now ready to build and run our project docker image, but wait what about the project files, well I’m getting to that actually let’s go ahead and create a simple test file for our example:

In my case I will create a folder and file example project, in practice we would use our project local repository copy.

Filename: ../demac-docker-example/public/index.php

<?php phpinfo(); ?>

We will also need to create an empty file inside the public folder called .htaccess-combined

And now we build our project image:

docker build -t amacgregor/localhost_com .

Finally we can go ahead and run our newly created image using the following command:

The above might look intimidating at first but let’s break it down:

-v /home/amacgregor/Projects/DemacMedia/demac-docker-example/public:/srv/www/localhost.com/public_html

With this option we are telling Docker to mount a directory from the host filesystem (the one that we just created) into the vhost root directory, this way we can reuse our local project files and test changes on the fly.

-p 127.0.0.1:80:80

All we are doing here is exposing the port 80 to our host machine port 80, please note that if you have any software like MAMP or Apache already running this won’t work and you will need to change the host port.

Finally, we can verify everything is working by loading localhost.com in our browser, if everything worked correctly we should see the PHP information page.

Feedback

This is the first time I ever worked with docker. Feedback and hints on how to improve are more than welcome. And if you find this useful let me know.

Design Patterns in PHP: Adapters

Sun, 25 Jan 2015 09:53:16 -0500

The adapter pattern also referred as the wrapper pattern, I find that wrapper is a more fitting name since it describes clearly what this pattern does; it encapsulates the functionality of a class or object into a class with a common public interfaces.

In software engineering, the adapter pattern is a software design pattern that allows the interface of an existing class to be used for another interface.[1] It is often used to make existing classes work with others without modifying their source code.

Adapters are one of the easiest patterns to comprehend and at the same time one of the most useful ones. Strangely enough documentation and examples for this particular pattern seems to be somewhat lacking in the PHP world (at least as far as the research for this article took me) so I’ll try to provide examples that are closer to real world usages.

But first, we need to go over some basic concepts about this pattern. As with any pattern, the adapter pattern has multiple participants:

Client: The client is a class or object that wants to consume the Adaptee public API.
Adapter: The adapter provides a common interface between the adaptee and its clients.
Adaptee: The adaptee is an object from a different module or library.

Another advantage of the adapter pattern is that allows us to decouple our client code from the adaptee implementations. Let’s see what that means in the following example:

In our example we will work with a notification manager class:

Although the previous example is oversimplified, it illustrates clearly were adapters can be useful, so let’s breakdown the problems with the notificationManager class.

The class knows too much about each notification type implementation.
If any of the notification classes change, we have to change our NotificationManager class code.
Adding a new notification type requires modifying the NotificationManager class code.

Regardless of the notification service being used we know that our notification class should do two things:

Accept the data from the NotificationManager
Send a notification

How the data is parsed, and the notification send is completely up to the service implementation; so based on that we can start by creating a NotificationInterface class:

The notification interface will be implemented by each and one of our notification adapters:

Based on the code above we can now rewrite our NotificationManager class in the following way:

Doesn’t that look better? As long as the notification services implement the NotificationInterface the notification manager doesn’t need worry about the specifics of each implementation.

However, there is still a problem with the NotificationManager class:

Adding a new notification type requires modifying the NotificationManager class code.

Fortunately, this can be solve easily if we use the Dependency Inversion Principle and since this is a bit out of the scope of this article, I won’t go into much detail about the specifics of this principle.

In object-oriented programming, the dependency inversion principle refers to a specific form of decoupling software modules.

Thanks to the use of adapters for the notification services, we can decouple of the notificationManager even more.

Summary

Adapters are extremely useful when combined with SOLID design principles and help developers to write cleaner and more maintainable code.

WTF are: Service Contracts (Magento 2 Edition)

Mon, 12 Jan 2015 00:00:00 -0500

Now that Magento2 beta has been officially released, you are probably wondering what the hell are Service Contracts and more importantly why the hell do you need them.

But before we jump into the nitty-gritty of service contracts let’s get this out of the way:

Yes, Service contracts are nothing more than a set of interfaces used to define the public api of a module.

Now, is important to clarify that by API we are talking about the set of interfaces that a module provides to other modules to access their implementations.

Service Contract Architecture

In Magento the service layer provides a formal contract between a client and the service provider. This form of contact allows evolution of a service implementation without affecting the client.

The Service layer allows modules to provide a well-defined public API and effectively hiding the business logic and preserving data integrity.

Magento 2 service layer is comprised of two types of interfaces, Data Interfaces and Service Interfaces. Data Interfaces are immutable data objects that allow to preserve data integrity by implementing the following patterns:

They define only constants and/or getters, meaning they are read-only.
Getter functions must have no parameters.
A getter function must only return one of the following objects types:
- A simple type (integer, string, boolean)
- An Array of a simple type
- Another Data interface
- Getter functions cannot return mixed types.
- Data interfaces can only be populated or modified by using data entity builders.

Service Interfaces, on the other hand, are in charge of providing a stable set of public methods that can be used with by the clients regardless of the kind (controller, the web service, other Magento modules)

Service interfaces differ a bit from their Data counter parts; we have three subtypes of service interfaces:

Repository Interfaces
Management Interfaces
Metadata Interfaces

Repository Interfaces

Repository interfaces are used to provide access to the persistent data entities, all this means is that they provide a specific set of methods interact with the data objects.

Repositories abstract the way data is mapped out to an object, this way the data storage(database) could be anything, and could change without affecting the clients (other modules).

Repository interfaces should implement the following methods:

save
get
getList
delete
deleteById

For each data entity, we have a corresponding data interface. But, if that wasn’t clear enough, let’s go ahead and see a couple of examples of how Magento2 makes use of Repository interfaces.

Ok, so that the interface definition but how is used? Let’s take a look at another class from the customer module.

For now let’s not worry about how the customerRepository object is instantiated (we will deal with that later on another post about the objectManager) but how is being used, in particular we want to look at the following function:

Why is it Great?

With Magento 2 use of the repository pattern, we could add a new customer extension, that changes the model and the resource model, to let’s, say save customer date into a MongoDB and as long that new customer model implements the necessary service interfaces we can be certain that we are not breaking the res of the application.

I will cover the rest of the interface types once the Magento2 documentation is updated and there are more details on their specific responsibilities.

Watch: We Strongly Recommend You Write Best Practices

Wed, 01 Oct 2014 00:00:00 -0400

What to watch this week, Lauren Rother from PuppetLabs give us their best practices to write best practices. A great set of recommendations for writing and mantaining your best practice documentation.

Writing best practices documentation is definitely an art, but that doesn’t mean a little science can’t help us along. Through some trial and error, I’ve uncovered some tenets of writing engaging, readable best practices docs. I’ll walk through a bit of my path to discovery as I highlight and give examples of successful and (enthusiastic) reader-approved best practices documents.

The Async Software Development Manifesto Revised

Mon, 29 Sep 2014 00:00:00 -0400

The ‘Async Software Development Manifesto’ has been making the rounds on sites like HackerNews and /r/programming; and while I don’t agree entirely with all that is proposed, some of the points made by the author made so much sense that I felt motivated to break them down and make some contributions of my own.

Async Software Development Manifesto

The manifesto appears to be inspired by the ‘Agile Manifesto’ and it advocates for a different approach and tools, in direct opposition with the Agile methodology. The following principles are emphasized:

Use Modern tools
Meetings only as a last resort
Flexible work environments
Document everything

Now, even as principles some of those are very generic and to be honest quite vague; so in order to understand what the author meant by each of those principles we need to break them down further:

Use Modern tools

On this point the Async Software Development Manifesto, which from now I will refer as the ASDM (since I’m getting tired of typing the whole thing) advocates for the use of modern tools like Bitbucket, Github, or Gitlab. The reasoning behind this is that those tools posses the following qualities:

Each deeply integrates version control with issue tracking.
Each offers rich prioritizing and assignment features.
Each wraps all that up into a slick user experience that anyone on the team can use, including nontechnical people.

The point that seems to be made is the importance of having tools that integrate with work-flow and reduce the friction for the developers and everyone involved. In that sense tools like Bitbucket or Github certainly are effective at integrating and issue tracking system, and a version control system all tied together by a common interface that technical and non-technical people can share.

However, the argument needs to be made against using a single tool like these; not every company follows the same work-flow and not all the clients are willing to learn how to use issue tracking systems like the ones mentioned before.

In some cases letting clients directly raise issues, without any kind of filter can be contra-productive and generate unnecessary work for everyone involved.

The second point that I want to make is the use of the word modern, old or new, trendy or not; those should not be a factor when selecting tools and methodologies to use for your team and work-flow.

Modern also implies that any tools that have been lying around for a few years are obsolete and to be avoided, but that is hardly the case some popular tools like VIM have been around for a few decades and it’s not showing signs of going away any time soon.

So in this case, I think the principle should be revised to:

Use tools that work for you

Meetings only as a last resort

According to the ASDM “meetings are very costly to your business” due the time that they take away from developers and the cost of constant interruptions, several articles like programmer interrupted, why you shouldn’t interrupt developers and the high cost of interruptions deal with the same concept; there is a cost that hidden when interrupting developers; so I agree with this part of the principle.

However, meetings specially the quick kind like daily stand-ups still have place, meetings shouldn’t be the last resort however meetings must be use smartly:

Interruptions should be minimized.
Meetings should be meaninful and to the point.
People should be hold longer than needed.

For example at Demac Media we apply this principle by doing morning stand-ups, but only on Mondays, Wednesdays and Fridays; meetings are also kept sort usually under 15min and developers are not hold longer as soon as they are done they can leave the meeting.

The main point is to provide a quick checkpoint for the team, make sure everyone is on track, and giving an opportunity for jr developers to raise any issues or problems they might be thus avoiding miscommunication. There is value to be found on effective meetings, when meetings have a clear purpose they can actually minimize and prevent problems.

There is however a valuable point to be made about constant interruptions and their attached cost, I have found that the following measures are effective to minimize interruptions while still following the concept of Async Software Development:

Instead of walking over to another developer to ask a ‘5min’ question, write an email. Writing an email forces you to think about the problem and what you have done to try solving it. This is basically ‘Async Rubber Ducking’ chances are that the asking developer already knows the answer to the problem, but is needs to change the way he is looking/thinking about the problem.
Define core hours, during core hours you should avoid interruptions (meetings, walking over, etc) the core hours should be a large chunk of interrupted time during the day. Define core hours will help developers to have a guaranteed and predefined period of time, that they can use to get complicated tasks done.

So in this case, I think the principle should be revised to:

Use meetings wisely

Flexible work environments

Here the original ASDM goes into a few points of what does it mean to have a flexible work environment:

Adopt a hotelling policy at your office.
Don’t assign desks to anyone by default.
Anyone who requests an assigned desk should get to choose whether it’s a private office or in a communal space.

And while I wholeheartedly agree with all those points, I think we have to be careful of how much flexibility is allowed. Some developers will completely isolate themselves if allowed to; and writing code in isolation is never a good thing.

Be flexible with your process

Be mindful about any process or policy, if the process gets in the way of actually getting work done then the process needs to change.

Document everything

The original manifesto doesn’t go into much detail on this last point:

The better documented your work-flow, the less your workers will need to interrupt each other to seek out tribal knowledge.

And I must say I don’t disagree, having proper documentation is always positive, however is how you generate and maintain this documentation what can cause problems.

It all comes down to making the documentation process unobtrusive and frictionless, this will help the developers to feel more comfortable while writing documentation and it won’t interrupt their work-flow, that been said there is no better documentation than well written code.

Watch: Funny Tech Support

Wed, 24 Sep 2014 00:00:00 -0400

What to watch this week, Let’s have a laugh watching Wes Borg and his fanstatic routine on the perils of tech support.

This Is Why I never got into Tech Support.

Watch: The Myth of the Genius Programmer

Wed, 17 Sep 2014 00:00:00 -0400

What to watch this week, In this week video Brian Fitzpatrick and Ben Collins-sussman talk about the pervasive myth of the uber-programmer the know it all genius; why everyone secretly wants to be considerad a genius and how this can affect the collaboration and behavior.

A pervasive elitism hovers in the background of collaborative software development: everyone secretly wants to be seen as a genius. In this talk, we discuss how to avoid this trap and gracefully exchange personal ego for personal growth and super-charged collaboration. We’ll also examine how software tools affect social behaviors, and how to successfully manage the growth of new ideas.

Watch: What is Docker?

Wed, 10 Sep 2014 00:00:00 -0400

What to watch this week, Solomon Hykes explains the docker platform and how it can help developers to build and deploy apps more easily. Great for those that want to understand more about the docker platform.

Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications.

Watch: Magento Fireside Chats on Testing

Mon, 08 Sep 2014 00:00:00 -0400

What to watch this week, the Magento Fireside Chats on testing. On this series of hangouts Magento developers come together to discuss their different approaches to testing, covering a variety of tools like Selenium, PHPUnit, PHPSpec, Menta and more.

Episode List

Episode 1: In this episode Vinai Kopp will show how to mock objects in your test cases and Fabrizio Branca will give an introduction into Selenium testing with Menta.

Episode 2: In this episode Allan MacGregor and James Cowie will tell us about MageTest suite, BehatMage and MageSpec.

Episode 3: In this episode Max Pronko will talk about the Magento Testing Framework (MTF) and Lee Saferite will show us how to write unit tests with fixtures using EcomDev_Phpunit.

Episode 4: In this episode David Robinson will show us how to work with the advanced screenshot features of the Menta Selenium Testing Framework and Bastian Ike how to use Jenkins’ multi-configuration job to test Magento modules with multiple Magento versions.

Watch: How to Build a Framework

Sun, 07 Sep 2014 00:00:00 -0400

What to watch this week, an amazing talk by Max Lynch from Ionic, where we recalls his experiences while creating the Ionic framework. Great Insights for anyone looking to start an open source project regardless if is a framework or not.

Max Lynch presents at the 2014 Midwest Mobile Summit on how to build a successful open source framework.

Exploring Hack: Building a MicroFramework

Wed, 06 Aug 2014 00:00:00 -0400

As a full-time Magento Developer I deal with a very specific kind of PHP every single day; this can at time get rather boring since rarely one can apply new technologies or programming concepts. One of the new languages that keeps grabbing my attention is HACK, which is almost identical to PHP in terms of syntax but at the same time implements some of the more attractive features from statically typed languages.

Hack is a language for HHVM that interopates seamlessly with PHP. The barrier to entry for Hack is low. To get started, all that is needed is to generally understand the features that Hack provides and how to call the Hack type checker (hh_client invokes the type checker at the command line)

I have covered Hack previously on the following articles:

Learn by Doing

I honestly believe the best way to learn something is to get your hands dirty and make mistakes; so instead of writing dozens of post on the many new features of Hack and why they are awesome (in theory) let’s build something useful.

So to get started I’ve decided to build a micro-framework using HACK and HHVM, building a simple microframework should be a challenging enough task to illustrate some of the more interesting features of the language and at the same time it has an achievable goal so we don’t end on a never ending development cycle.

Let’s call our new framework something clever:

Meet Slash

Get it? Hack, Slash (it’s a clever name, shut up). Before we start writing any code let’s think a little about what kind of micro-framework we want to build. There are many amazing PHP micro-frameworks out there, just to mention a few:

Outside of the PHP world, there is Sinatra which is Ruby based and excellent example of a powerful micro-framework. Curiously enough Sinatra seems to be the source of inspiraton for many of these frameworks; so let’s not break tradition and use Sinatra as our main source of inspiration.

That means that Slash main purpose will to create RESTful applications, and as such the main component for a framework like this will be its router.

In the next post of this series we will go through the creating of the routing and the general structure of new microframework.

Swiss Army Knife Syndrome

Tue, 20 May 2014 00:00:00 -0400

I apologize beforehand if this article is more of a rant than usual. The inspiration for the “Swiss Army Knife Syndrome” came from my frustration in dealing with project managers, clients, and even other developers, that think in too much of a narrow, particular way. I call it the “Swiss Army Knife Syndrome”.

The Swiss Army Knife

The term ‘Swiss Army Knife’ is often used to describe a collection of useful items or tools that are able to perform well in multiple scenarios.

While this may be useful, there are risks to be aware of as well. A tool with too many moving parts can end up being completely useless! By trying to do everything, said tool might be great at nothing.

In my experience, the same problem can apply to software. More often than not, developers try to include a feature or piece of code just because it is “cool”; project managers will try to modify the scope in the middle of a project because “X” or “Y” feature will add more value; customers will request extra features or functionality because they read or heard they were “critical” to their business.

This ‘Swiss Army Knife Syndrome’ can take many shapes: scope creep, early optimization, and so on. But the root of the problem is how we perceive and value software, work, and the value attached to it:

More Features
=
More Value

In reality, and for most cases, the opposite is true. The more complex a piece of code or software gets, the less value it provides. A personal example that illustrates this concept was a simple, pivotal dashboard for Demac Media internal use.

The original application was simple: we needed a way of (1) seeing all the tasks assigned to a particular team and (2) to filter them by the current week or two week sprint-basically, a task aggregator with basic filtering.

I coded the simple concept over the course of one weekend. Upon showing this to my team’s project manager the following Monday, he deemed the application useful.

... but it would be so much better if ...

And that’s how Swiss Army Knife Syndrome begins: with a ‘but’. The tool was shared with other teams. Before they even started using it, we had a list of half a dozen features that were ‘needed’ or would add more value to the application. Suddenly, we had a number of requests well beyond the initial scope of the application.

Clear Purpose

Software needs to be as clean and as simple as it can practically be. To follow the knife analogy, good code should be like a chef’s knife. A chef’s knife has a clear and defined set of uses. A professional chef will use the right kind of knife for the job. This is how we should think about our code.

Do one job and do it well.

We find the same principle in software design, often referred as the Single Responsibility principle:

…The single responsibility principle states that every class should have a single responsibility, and that responsibility should be entirely encapsulated by the class. All its services should be narrowly aligned with that responsibility.

Conclusion

No company, project manager, developer, or client is exempt from falling for this faulty logic. We are inclined to think that having and doing more is equivalent to being better or having more value. Software should be elegant, and elegant code is where simplicity meets a good solution. Therefore, it is our responsibility as developers to ensure that every bit of code we produce is as elegant and succinct as possible.

Special Thanks to:

Mark Holmes - markholmes.io

Flexible PHP Development with PHPFarm

Fri, 18 Apr 2014 00:00:00 -0400

If you have been working with PHP for a while, chances are that you have come across with a project, extension or script that requires to be tested on multiple PHP versions, for simple CLI scripts this seems easy enough but what happens when you are working with complex applications, developing for frameworks or multiple versions of them ?

Let’s say that like me, a Magento developer that regularly develops extensions, there is a need to run on multiple versions of Magento. The initial approach that most developers take is to create a virtual machine with the right environment. For an application like Magento that means you need two virtual machines one for PHP5.2 and one for PHP5.3.

Or at least that used to be the case because PHP5.3 is officially being deprecated, now we have to include PHP5.4 into the mix; with out current setup that means creating another virtual machine setting up all our sites again, deploying our changes to each of the virtual machines.

Note: There are possible workarounds to avoid deploying multiple versions of the code across all servers and sharing the code across all virtual machines but that is outside the scope of this article.

This setup can quickly become cumbersome and it is not easily scalable: what happens if we want to test individual patch version or test code on all 3 versions, or add PHP5.5 depending of the host machine you use, it might not be able to run more than virtual machine at the same time.

I ran into all these problems while working at Demac Media where I constantly need to work on multiple environments and specially when testing extensions for commercial distribution. In order to work more efficiently I came up with the following setup than can be ran locally or on a single Virtual Machine if you are on windows.

Meet PHPFarm

Getting multiple PHP versions running side by side can be challenging and over the year devs have released multiple solutions like PHPEnv or the new VirtPHP, personally I use PHPFarm which works very well with my workflow and is extremely easy to use, as well working seamlessly with Apache.

The basic idea behind PHPFarm is that we can install several version of PHP side by side, in addition we can select one as current or reference specific version directly. Another extremely nice feature about PHPFarm is that we can run patch versions side by side.

Installation and Configuration

$ cd /opt/ $ git clone https://github.com/cweiske/phpfarm.git phpfarm $ cd phpfarm/src/

Note: In order for PHPFarm to work properly you should add the _/opt/phpfarm/inst/bin_ and _/opt/phpfarm/inst/current-bin_ directories to your path, the method for changing this will vary depending on your current OS.

So at this point we have everything that we need to install any PHP version, so let’s go ahead and install PHP5.3.0:

$ cd /opt/phpfarm/src/ $ ./compile 5.3.1

At this point PHPFarm will take care of downloading the source files for the specified version and compiling for our system, while compiling from source might be a little slower than using the distribution binaries, we have much more flexibility over the configuration and options if the installed PHP version.

Configuration

As I mentioned before we can control each individual version compilation options and default php.ini settings; both are controller by configuration files inside the src/ directory.

The default configuration options are in src/options.sh. And we can create version specific option files like:

custom-options.sh custom-options-5.sh custom-options-5.3.sh custom-options-5.3.1.sh

This structure give us very granular control over the PHP options, and this one the nicest features PHPFarm has to offer. Like wise we can do the same with the php.ini values:

custom-php.ini custom-php-5.ini custom-php-5.3.ini custom-php-5.3.1.ini

An example options and ini file might look something like the following:

Installing Multiple Versions of PHP

Previously, we installed one of the older versions of PHP5.3 but in order the get the hang of working with multiple versions of php, let’s go ahead and install the latest versions of PHP5.4 and PHP5.5:

$ cd /opt/phpfarm/src/ $ ./compile.sh 5.4.27 $ ./compile.sh 5.5.11

In order to confirm that everything was installed correctly let’s verify each of the php versions we installed works:

$ /opt/phpfarm/inst/php-5.3.1/bin/php -v $ /opt/phpfarm/inst/php-5.4.27/bin/php -v $ /opt/phpfarm/inst/php-5.5.11/bin/php -v

After running each one of those commands you should see each version’s full information, but is highly inconvenient if we have to type the full path to each of the php versions, right ? Well no worries PHPFarm actually has our back on this one, PHPFarm comes included with a command called with switch-phpfarm.

Let’s try it out:

$ switch-phpfarm 5.4.27 After running that we can go ahead and try to get php information by doing the following:

````

It’s as simple as that, switch-phpfarm takes care of creating the necessary symlinks need to run php; switching back to PHP5.3 is as simple as too:

````

Adding Apache to the Mix

So we have multiple PHP versions running side by side and we can execute code by calling the php version directly or by changing the current version, however this is not very useful, real world PHP development goes well beyond single scripts executed through the shell. We need to be able to execute and test complex applications, and for that, we need a web server.

Personally, I like to use mod_proxy_fcgi and PHP-FPM on my development and production stacks, however mod_proxy_fcgi is only available on Apache2.4 which unfortunately is not an option for some developers out there, but no need to worry we can still setup older versions of Apache to run specific versions of PHP per VirtualHost.

Note: I would still highly recommend using Apache2.4 and mod_proxy_fcgi, the module provides a lot of flexibility and allows you to segment parts of your applications, for example, running the backend and frontend of Magento using different PHP-FPM pools or even one running HHVM

For this article’s example purposes we will use mod_fastcgi in order to run different version of PHP per VirtualHost. The first thing we need to do is make sure that mod_fastcgi is installed and running; in Ubuntu this can easily be done by running:

$ a2enmod fastcgi actions suexec $ service apache2 restart

Next we need to tell Apache where it can find the configuration for the multiple fastcgi servers, open the apache configuration file located in /etc/apache2/apache2.conf and add the following lines at the end of it:

# Include FastCGI configuration for PHPFarm IncludeOptional cgi-servers/*.conf Now we need to create the cgi-servers/ directory and the corresponding file:

$ sudo mkdir /etc/apache2/cgi-servers/ $ cd /etc/apache2/cgi-servers/ Inside create a file named php-cgi-5.3.28 and copy the following content:

FastCgiServer /srv/www/cgi-bin/php-cgi-5.3.28 ScriptAlias /cgi-bin-php/ /srv/www/cgi-bin/

Next let’s create the file /srv/www/cgi-bin/php-cgi-5.3.28:

```` #!/bin/sh PHPRC=”/etc/php5/cgi/5.3.28/” export PHPRC

PHP_FCGI_CHILDREN=3 export PHP_FCGI_CHILDREN

PHP_FCGI_MAX_REQUESTS=5000 export PHP_FCGI_MAX_REQUESTS

PHP_FCGI_IDLE_TIMEOUT=5000 export PHP_FCGI_IDLE_TIMEOUT

exec /opt/phpfarm/inst/bin/php-cgi-5.3.28 ````

We also need to make sure the file is actually executable:

$ sudo chmod +x /srv/www/cgi-bin/php-cgi-5.3.28 Finally, we can go ahead and create a virtual host that points to the newly create php-cgi server:

Let’s now enable our VirtualHost and restart the Apache server:

$ a2ensite test.mydomain.com.conf $ service apache2 restart

Now let’s make sure our Virtual Host is loading the proper PHP correctly, to do this we can create a phpinfo script on the root directory of our VirtualHost, with the following content:

<?php phpinfo(); ?>

With this setup we can install and use as many PHP version as we want, side by side and without any conflicts.

Final Comments

Although this setup is tailored to my work flow, I’m sure that other developers will find useful or at the very least will give you ideas to improve your own development environment and work flow.

Also if you spot any errors, have any suggestions or want to share your own setup please feel free to leave a comment on the section below.

Is HHVM/Hack the new face of PHP?

Tue, 08 Apr 2014 00:00:00 -0400

HHVM has been generating a lot of buzz and excitement in the PHP world recently and if you haven’t heard about HHVM yet you should check my first two posts about it.

Why Should I Care?

That’s a really good question why you should care about HHVM, after all is being developed by Facebook (and Facebook is eeeviiiil right?); well the case is that HHVM is – at least in my humble opinion – the coolest development going around the PHP world right now.

HHVM could change (is already at some extent) the way developers see and think about PHP performance, and how PHP should work. Even if you don’t care and never plan to use HHVM or HACK in any of your projects the mere fact that they exist is a good thing for the PHP community.

Competition, that’s the first thing that HACK and HHVM brings to the table, ideally we should see some features from HACK reimplemented into later versions of PHP, driving development of the standard PHP language further.

HACK Will Never Replace PHP

Nor it should, seems one the most popular topics being used on Hack related articles is about how it could or could not replace PHP (see Sitepoint 2 or Sitepoint 2 ) but in my opinion HHVM/HACK was never meant to be a PHP replacement; it was designed to live side by side with PHP even in the same project.

Heck, you could even have PHP and Hack code running in the same class file and by setting the correct ‘Mode’ parts of the code can be type checked and some can be ignored and treated as regular PHP; Hack’s interoperability with PHP is one of its strongest features and truly lowers the entry curve for all PHP developers.

Final Thoughts

Right now the biggest criticism against HHVM/HACK is that Facebook could just drop the project in the next couple of years, a concern that can easily be put to rest by the fact that Facebook own massive code base(which is in the millions of lines of code) is now running almost on its entirety on HACK.

HHVM/HACK are a refreshing look at the PHP language, is my opinion than even in the bleakest of scenarios there is already a positive outcome from HHVM/HACK coming into existence, as for how far will HACK influence the standard PHP language that is yet to be seen and it will depend greatly on the PHP internals group.

Working with Psysh

Thu, 03 Apr 2014 00:00:00 -0400

I previously mentioned Boris, a terrific REPL for PHP, and while Boris is a great and functional REPL, is not the only available.

Psysh is actually more than a simple REPL it’s also an interactive debugger; which means you can say goodbye to the endless barrage of var_dump() and die() statements.

But do we really need another REPL for PHP, well honestly we could probably get by with the solutions currently available however Psysh has an extremely interesting Ace under the sleeve, it can also function as a realtime debugger.

Regardless of the mode on which is running Psysh can offer a lot of information about our current application state and code. For example the PHP documentation is available by using the doc command:

We can also get more information about our current application objects and their properties:

Psysh has also been integrated with several popular frameworks like:

If you are looking for a powerful REPL to integrate with your PHP project, Psysh might the right tool for the job; with powerful features and an integrated debugger Psysh has a lot of value to offer to any project.

Hello Hack

Sun, 23 Mar 2014 00:00:00 -0400

Recently Facebook unveiled Hack, a new programming language that aims to provide developers with the tools to write and ship code quickly while catching errors on the fly.

What is Hack?

Hack is as programming language designed to work with HHVM and the same time it works seamlessly with PHP as Facebook lead devs:

Hack has deep roots in PHP. In fact, most PHP files are already valid Hack files. Facebook

So does that mean that Hack is just a faster more efficient PHP implementation, right? Wrong, Hack is much more than that, the language brings features that are normally found in statically typed languages to the dynamically typed world of *PHP**.

Hack reconciles the fast development cycle of PHP with the discipline provided by static typing, while adding many features commonly found in other modern programming languages… Facebook

Static typing

In a dynamically typed language like php variables types and its values are both mutable, this means that a variable can start like a null then take a string value and finally change to an integer all inside the same local scope. While this is great for writing scripts and simple applications; as things start to scale the codebase will become harder and harder to maintain.

With millions of lines of codes and hundreds of developers working on the same code base on daily basis this is exactly what happened at facebook, which was originally developed in PHP and now is running almost entirely on Hack.

The main and so far most noted feature in Hack its is static typing. In Hack you can annotate variable, function return types and, the function member variables. This alone can help tremendously to prevent errors like the following:

On the snippet above dbfetch() could return a null value instead of the object that our code is expecting; PHP errors can only be caught at runtime, but with Hack we can run a type checker against the code and catch this type of errors early in the development cycle.

More than static types

There is more to Hack than static types among the many features the language implements we have:

Collections
Tuples
Traits
Lambda Expressions
Generics
Async Programming

And that’s only a few of the features that the language has to offer, Collections are particularly interesting since the come as replacement for PHP Arrays, Traits and Trait Generics are a welcome addition, Async programming gives developers the option to develop non-blocking code.

Summary

There is a lot that Hack has to offer to the PHP community, personally I think Hack is a welcome addition to the PHP Universe. Although the general reaction has been mostly positive there has been some criticism and negativity towards Hack, and Facebook decision to stick with PHP.

And while Hack stability and adoption has yet to be put to the test, the mere fact that something like Hack exists is beneficial for the community and PHP as a language, there is a very good chance that many of the feature hack has could be ported to future PHP versions.

Finally, there is one massive strength that in my opinion HHVM and Hack currently have on their favor and that is the seamless interoperability between Hack and PHP; developers can take a large codebase and slowly start rewriting the code in Hack class by class even as far as individual functions while the rest of the application is still running on PHP.

I was able to put this to the test by hackifying a small Magento extension and have it run successfully along with Magento.

HHVM In The Wild

Sat, 22 Mar 2014 00:00:00 -0400

HHVM is taking the PHP world by storm, and as PHP developer there many things to be excited about HHVM.

The following list shows how developers are using HHVM in the real world in very different scenarios.

Additional Reading

If you using HHVM in production or in a way where you can see great performance gains and want to share with us? Then drop me an email at info (at) allanmacgregor.com or leave a comment in the section below.

Exploring Traits

Mon, 17 Mar 2014 00:00:00 -0400

Among the new features and fixes that come with PHP5.4 we have the addition of traits to the PHP language, PHP is a single inheritance language this means that classes and only inheriting from single parent class, in practice this complicates code organization and can lead to code duplicity.

Languages like C++ or Python manage this problem by allowing inheritance from multiple classes, Ruby in the other hand uses Mixins to address this issue. Regardless of the technique the problem remains the same; Traits are another approach to this problem and are commonly used in the languages like Perl and Scala.

Although PHP5.4(and Traits) has been around since early 2012, a lot of php developers might not be familiar with the concept and power behind Traits; In this article I want to explore traits, their usage, advantages and disadvantages.

PHP and Multiple Inheritance

The main reason behind PHP lacking multiple inheritance is the lack of concensus of how to solve the diamond problem which arrises when multiple inheritance is implemented. The diamond problem can be described as an ambiguity in multiple inheritance OOP; but let’s see an example to understand what th is means.

The diamond problem gets its name from the shape that the class inheritance takes in the particular situation where:

… two classes B and C inherit from A, and class D inherits from both B and C. If there is a method in A that B and/or C has overridden, and D does not override it, then which version of the method does D inherit: that of B, or that of C? wikipedia

So assuming PHP would allow for multiple class inheritance(which it doesn’t) the diamond problem would look something like this:

Whoops at that point (if PHP actually had multiple inheritance) we would get an error saying, since PHP wouldn’t know which roar() implementation to call.

This in essence is the problem with multiple inheritance, (un)fortunately PHP being a single inheritance language doesn’t have this problem and with the PHP5.4 we can implement similar functionality to multiple inheritance model.

A Trait Traits

Traits are a mechanism for code reuse in single inheritance languages such as PHP. A Trait is intended to reduce some limitations of single inheritance by enabling a developer to reuse sets of methods freely in several independent classes living in different class hierarchies php.net

An easy way to conceptualize traits would be to think of them as an interface with an implementation. As we mentioned before single inheritance has been part of PHP OOP (Object Oriented Implementation) over the years this more than one developer has been frustrated by this characteristic when trying to write code that is both clean on a complex system.

With Traits we can reuse functionality from other classes without having to extend them. Now, this might sound like multiple inheritance but the traits implementation is more akin to horizontal code reuse; Inheratence in the other hand is considered vertical code reuse.

As we mentioned before Traits are similar to Abstract classes, for example they cannot be instantiated on its own. Let’s look at the following example:

Now, if you are thinking that could you have done the same by creatin a Cat class that extends the Animal class and extending our Tiger class from there; take the following example into consideration:

Wasn’t that cool? Try to do that with single inheritance.

The best part about traits is that it makes sense from a structural point of view, think about it Cats and BigCats share many traits among them but not all of them, for example we could break it down even further:

Conclusion

Traits are an incredible addition to the PHP language and we have only started to touch the surface, in future articles will go over some of the more advance usages and caveats of using Traits.

Laravel Learning Resources

Sat, 15 Mar 2014 00:00:00 -0400

For the last couple of months I been working with Laravel and learning a lot, here are some of my favorite resources:

Books

Blogs

Screencasts and Podcasts

Boris The Missing PHP REPL

Thu, 13 Mar 2014 00:00:00 -0400

REPL(read-eval-print loop) can be great tools for quickly testing concepts, experimenting and getting quick feedback when learning a new language. Many languages and frameworks provide some sort of REPL like the rails console or laravel’s artisan tinker.

PHP by itself has the interactive shell that can be invoked with:

php -a However the PHP Interactive Shell lacks several features that other modern languages have in their REPL’s; like proper error handling, multi-line support and result output by default.

Meet Boris

Python has one. Ruby has one. Clojure has one. Now PHP has one too. Boris is PHP’s missing REPL (read-eval-print loop), allowing developers to experiment with PHP code in the terminal in an interactive manner. If you make a mistake, it doesn’t matter, Boris will report the error and stand to attention for further input.

Boris, was developed by Chris Corbyn a PHP developer that after making the transition to Ruby was disappointed by the lack of a true PHP REPL.

After playing around with Boris for while, I can honestly say that it has the right to call itself ‘The Missing PHP REPL’; not only provides the basic features that you would expect from a standard REPL but also:

Output customization.
Configuration files.
Start Hooks.
Loading larger applications.

On top of that Boris can be easily installed or included in any project using composer.

All these features make Boris a powerful and flexible tool regardless if you are just using it to test quick snippets or as part of a large application.

Magento Online Hackahon 2014

Sat, 08 Mar 2014 00:00:00 -0500

This past weekend the Magento Community hosted the first Magento Online Hackathon, here is a quick recap of my experience.

The Community

First and foremost a big shout out to the Magento Community, you guys are awesome. Very few other communities could and have organized themselves to run a Hackathon this size let alone an online one; It’s truly amazing to see people from so many countries and in different timezones trying to work together.

If anything at all was achieved in this event was to show that what makes Magento so unique and successful it’s community and the people in it.

The Hangouts

Google Hangouts! What can I say? Although not all the teams used them, they where a great tool to facilitate communication with the teams, it also allowed us the organizers to easily reach people and broadcast the progress of the projects to anyone tracking the event.

The Teams

Organizing the teams was a topic that the organizers spend a lot of time discussing, We where unsure of what would be the best approach to determine the team sizes, allocating the developers, and managing people switching between projects.

Should we limit the teams sizes?
What happens if people wanted to switch projects.

Normally in an offline hackathon rules are a bit more strict and people are easier to organize, doing an online hackthon presented many challenges in terms of communication, organization and participation.

In the end it was decided that rather than enforcing the team allocation, we would use the votes registered at magento.corehack.de to build a initial team rooster based on each persons vote, this initial rooster is just a suggestion for the initial team organization and participants could switch and even start new projects.

In the end, I’m happy to say that the teams where more than capable of organizing themselves and there very few if any problems at all.

The Projects

The Magento community proved that it can be very productive, and with over 20 projects there was plenty of work to do; here are some of the projects that came up as result from the online hackathon:

Magento and HHVM

Mon, 17 Feb 2014 00:00:00 -0500

If you are developer or system administrator working with Magento running one or more medium sized stores, chances are that you are familiar with the many challenges of optimizing and scaling Magento.

Magento is (in)famous for its performance, specially when scaling to a large numbers products, transactions or even catalog rules, seasoned Magento developers have probably hit at least one of this performance bottle necks more than once.

From the sysAdmin perspective, Magento can be a beast eating away resources and with many bottlenecks and paths to solve them:

Database Optimization
Cache Optimization and Architecture
Apache vs Nginx
mod_php or php-fpm
OPCode Caching

And while all the optimizations help, in the end there is a major performance bottleneck that is not as easily surpassed and that is PHP performance, since PHP is an interpreted language there is price to pay in terms of speed of execution and overall performance.

As large PHP like Magento try to scale, this performance price gets more noticeable and harder to solve, sure we can always throw more hardware, setup load balancing and fire up a reverse proxy cache; but that will only mask the issue.

HHVM

Facebook, believe it or not; is runs on PHP and of course it had to deal with the same problems that we talked about before. To solve them Facebook took and interesting approach by creating HHVM (HipHop Virtual Machine).

HHVM (HipHop Virtual Machine) converts PHP code into a high-level bytecode (commonly known as an intermediate language). This bytecode is then translated into x64 machine code dynamically at runtime by a just-in-time (JIT) compiler. In these respects, HHVM has similarities to virtual machines for other languages including C#/CLR and Java/JVM.

According to Facebook, they have seen between 6x to 9x the improvement on the performance and speed of their servers, this is big news for any PHP application and not only Magento.Having Magento run on HHVM would drastically improve the speed and performance of the actual application without relying solely on caching.

Currently, there are still many bugs on certain parts of the HHVM libraries that prevent Magento from fully running. However a lot of progress has been made recently to solve many of these issues; right now the main force behind all the bug fixes to get Magento and HHVM running is Daniel Sloof.

Daniel Sloof is a Magento Developer located in Netherlands; has worked with Magent since 2010 and can be contacted at daniel@rubic.nl or on twitter @daniel_sloof

Daniel has been working for the last few months on HHVM and OSX compatibly, as well as the HHVM issues with simple_xml.

As the time of this article publication, Magento can run on HHVM, but should still be approached with caution for production environments.

Getting Things Up And Running

So you want to run Magento on top of HHVM? If that’s the case you’re in luck, let’s walkthrough the process for setting up, installing and running Magento+HHVM.

Notice: While the advantages for running Magento on HHVM are many, HHVM is still under heavy development for that reason I would advise caution when running production projects on HHVM.

Installing HHVM

To run HHVM we will have to compile from the source, since HHVM is still under active and heavy development this is the best way to guarantee that we will get the latest and greatest version.

This article is not a detailed walkthrough for installing HHVM, if you get stuck or want to learn more read First steps on HHVM or additional you can visist HHVM Wiki for environment specific instructions.

We will need to get a copy of the HHVM repository:

Next we will need to configure and compile HHVM:

Testing HHVM

At this point we have a working hhvm binary that we can use to run php scripts, let’s verify that our hhvm instance works by running, execute the following command:

$ /path/to/my/installation/hhvm/hphp/hhvm/hhvm --help If everything is working correctly we should see the following output:

Now if you want to be able to run HHVM easily from any location you can add it to the PATH by creating a symlink:

$ ln -s /path/to/my/install/hhvm/hphp/hhvm/hhvm /usr/bin/hhvm

Configuring HHVM for Magento

The next step will be setting up the basic configuration file for HHVM and running our first test:

HHVM works around .hdf configuration files, HDF stands for Hierarchical Data File although in my opinion Hiphop Definition File would also make sense.

The definition files contain all the configuration for a specific HHVM instance they are somewhat equivalent to the apache configuration files. Most of the settings will be self explanatory, however I think it would be helpful if we breakdown the each of the configuration settings and we go into detail.

As we can see in the example file, there are 4 main configuration nodes:

Server
Eval
VirtualHost
StaticFile

#### Server

The server node contains configuration that is specifically used when hhvm runs in server mode (surprising I know!), among these settings we have options like port, listening ip, hostname, number of threads to spawn and so on.

The are 2 settings to which we should pay particular attention for our Magento installation, SourceRoot and DefaultDocument.

SourceRoot allows us to specify the Magento root folder, this would be the equivalent of Apache DocumentRoot.
DefaultDocument allow us to set the index file to be used automatically, this would be the equivalent of Apache DirectyIndex.

Eval

The Eval node controls the JIT(Just In Time) Compiler settings in this case the only thing that we need to know is that it should be enabled to get the full benefits from HHHVM.

VirtualHost

The Virtual host node allows us to set certain settings by virtual host like ServerName and ServerVariables in this specific case we need to set the rewrite rules to work with Magento rewrite settings, in order to resolve the assets and urls correctly.

StaticFile

The StaticFile node as the name describes allow us to configure how HHVM will handle and server files like css, js and image files.

Now, that we have a working configuration file we can run our first test on HHVM.

Running Magento with HHVM

Finally, we can get Magento up and running with HHVM, let’s start HHVM using the following command:

$ hhvm -m server -c example.hdf At this point, we should see our Magento website if we go to http://localhost/.

Measuring the Speed

Finally, let’s run a quick speed test with siege and compare the speeds of HHVM built-in server VS Apache2 + PHP-fpm:

Apache2 and PHP-FPM

```` $ siege -c20 -t1m localhost ** SIEGE 3.0.1 ** Preparing 20 concurrent users for battle. The server is now under siege… Lifting the server siege… done.

Transactions: 436 hits Availability: 100.00 % Elapsed time: 59.14 secs Data transferred: 1.89 MB Response time: 2.14 secs Transaction rate: 7.37 trans/sec Throughput: 0.03 MB/sec Concurrency: 15.77 Successful transactions: 436 Failed transactions: 0 Longest transaction: 3.06 Shortest transaction: 0.51

````

HHVM Build In Server

```` $ siege -c20 -t1m localhost ** SIEGE 3.0.1 ** Preparing 20 concurrent users for battle. The server is now under siege… Lifting the server siege… done.

Transactions: 1950 hits Availability: 100.00 % Elapsed time: 59.96 secs Data transferred: 9.32 MB Response time: 0.13 secs Transaction rate: 32.52 trans/sec Throughput: 0.16 MB/sec Concurrency: 4.17 Successful transactions: 1950 Failed transactions: 0 Longest transaction: 0.47 Shortest transaction: 0.07

````

Notice: The previous tests are by no means comprehensive or accurate, and more than anything they try to showcase the speed difference between the two setups.

Summary

We covered a lot of ground here, and we learned how to get Magento up and running with HHVM Built-in server, that being said I feel that is important that we clarify a few things.

While this article uses the hhvm built-in server, it is important to note that it is not the recommended method of using HHVM. For production setups, Apache/Nginx + FastCGI + HHVM is the recommended setup. Here are some of the reasons why FastCGI should be the prefered method:

The Built-in webserver will be eventually deprecated.
You can keep using all the existing rewrites.
Static assests are served slightly faster by something like Apache or Nginx.

However, for the scope of this article and learning purposes the built-in server is a good introduction to HHVM and the general setup and configuration, in a further article we will cover the production setup with FastCGI

Design Patterns in PHP: Singletons

Mon, 27 Jan 2014 00:00:00 -0500

The singleton pattern is used to restrict the instantiation of a class to a single object, which can be useful when only one object is required across the system.

Singletons are designed to ensure there is a single (hence the name singleton) class instance and that is global point of access for it, along with this single instance we have global access and lazy initialization.

A basic Singleton implementation will look something like the following example:

In the previous example all three variables will point to the same object. The first call will instantiate the object while any subsequent will only return the instantiated object.

In computer programming, lazy initialization is the tactic of delaying the creation of an object, the calculation of a value, or some other expensive process until the first time it is needed.

Let’s address elephant in the room, and talk about why Singletons are considered an Anti-pattern by many developers, although this is going to highly depend on the framework and language used; for PHP Singletons are almost universally considered an Anti-pattern.

Singletons are considered by many to be an anti-pattern, Anti-patterns are design solutions that are usually ineffective and present a high risk of being counter productive.

Singletons the spawns from Hell?

If you have read about Singletons before you are probably wondering what the hell am I doing?

Singletons are evil, an Anti-pattern and should never be used! Well that’s exactly what I want to address in this article; saying Singletons are Evil!! is not enough, we need to understand why Singletons shouldn’t generally be avoided.

There are several reasons why Singletons are considered an Anti-pattern, let’s take a look at some of those reasons:

Single Responsibility Pattern

This first problem that we encounter when using Singletons is that their usage breaks the Single Responsibility Principle.

Singleton objects are responsible of both their purpose and controlling the number of instances the produce, while the Single Responsibility Principle states that:

… every class should have a single responsibility, and that responsibility should be entirely encapsulated by the class.

Hidden dependencies

What are hidden dependencies and how is that relevant to Singletons?, well if you read my previous article on Dependency Injection we saw how to pass dependencies as parameters to a function.

Any parameters accepted by a function are called visible dependencies, on the other hand if a function requires something else to operate that is referred through a global variable – read singleton – then that dependency is considered hidden.

Now this presents because there is no way for a third party to know about this hidden dependencies without taking a look at the actual function implementation.

A visible dependency is a dependency that developers can see from a class’s interface. If a dependency cannot be seen from the class’s interface, it is a hidden dependency. jenkov.com

Testing

Along with these problems we have an issue when trying to Unit Test our applications. Proper Unit Tests should run independently of things like database, Singletons make unit tests difficult if not impossible since they have a global state.

That means that a singleton will stick around between Unit Tests once its instantiated, this could and will cause tests to unexpectedly influence each other.

There are workarounds that take care of ‘cleaning up’ the singleton after each test is run, however I find this contra-productive and messy.

It’s not all bad, is it ?

They can’t be all bad, can’t they? Well let’s try to argue in favor (sort of) of Singletons and some cases where they might be useful:

Debug Logging: Almost all developers will agree that a way to debug logging should be available for every function and part of the code. A singleton could serve this purpose without harming readability, testability or maintainability.
Filesystem and Database Access: The argument can be made for Singletons proving access to the filesystem and database, now this might work if you need a single global filesystem or database access point it trades flexibility and testability for modicum amount of convenience.

Conclusion

Singletons, Anti-patterns, and patterns in general are not good or bad; what makes a Singleton an Anti-pattern is not the pattern itself but how often is poorly implemented and how easy it is to do so.

Any pattern can become and an Anti-pattern if incorrectly implemented, which happens often. That being said, when speaking of modern PHP and modern frameworks it is hard to make case in favor of Singletons and personally I don’t see any advantages that out weight the many downsides that come with this pattern.

As a final note I understand that Singletons are a controversial and polarizing topic, if you have an opinion, would like to complement any of the points made in this article or even better make a case in favor I would love to hear your comments below.

Design Patterns in PHP: Using Factories

Sun, 19 Jan 2014 00:00:00 -0500

The factory pattern is one of the most commonly used Design Patterns, it is useful when we need to choose between several interchangeable classes at runtime. By using factories we are also separating the creation of an object from the actual implementation.

We can define a factory as a simple design pattern that give us a convenient way to instantiate objects. A factory is capable of creating different types of objects without necessarily knowing what type of object is actually being created.

Factories try to address the problem of tight coupling in large application, with a factory instead of calling new directly, we use our factory class to take care of creating of the object.

There are several implementations and variants of design pattern among of which we have:

Simple Factory
Abstract Factory
Factory Method

On this post we are going to focus on the Factory method, which is closer to the original Gang of four definitions and has more practical applications, as a side note the Simple Factory is normally not considered an actual pattern by many developers.

Personally I do consider the Simple Factory to be a pattern, just a rather simple one; and I’m not going to start a debate about what pattern is or is not.

As you can probably gather from the pattern name, a Factory Method works by using a class method for creating and instantiating the required objects.

When to use a Factory

Know we have a general idea of what a factory class is and what is used for, but when is it appropriate to use it? Factory classes are generally used when:

A class cannot anticipate the object types that needs to create beforehand.
We want to encapsulate the logic for instantiating complex objects.
We want to reduce tight coupling between our application classes.

Enough talking let’s create some practical examples of factory class usages.

Show Me The Code!

Let’s imagine that is our first day at the Ikea Product Factory and we have been tasked with maintaining and updating the code; as you can imagine, this being Ikea, there are thousands of different products and products types. For the sake of argument, let’s imagine that every single product has its own unique class. So we would have something like this:

So as in the code example above, we have a Product abstract class that implements the construct function and the getters and setters that are shared among all the product classes. For each product we have a class inheriting from the Product abstract class.

You also notice when reading the existing code is that there are hundreds of calls to instantiate each individual product class. Like so:

A clearer example of how this can become problematic is if we had a controller running on http://factory.ikea.com/product/create/{product_type}, this controller is in charge of instantiating a new product objects based on the $product_type provided, validating and adding the information posted and saving the product.

We are going to ignore the logic for validating the data and saving the product, right now we only care about how that product object is being instantiated without the use of a factory:

What’s the biggest problem with this code? Simply put since our controller doesn’t know beforehand what type of product it’s going to create, our fictional Ikea developers used a switch statement to handle the right class instantiation.

Now you might be thinking that doesn’t look so bad? It gets the job done, right? Well remember that it is our fictional IkeaFactory and we have hundreds of different product classes; meaning that if this was a fully implement class we would have hundreds of different cases on our switch statement.

The previous example is not only messy but hard to maintain too, what would a happen if we suddenly added a new parameter to our __construct() function? You guessed right; that would mean updating every single instance where we call new on any of the product classes; that would mean a few hundred lines changed on our example controller function.

Let’s see how implementing a basic factory can help us write better and more maintainable code. First we need to create a new class called ProductFactory

And now lets refactor our controller action to use our new factory:

Lets that sink in for a second, we just replaced hundreds of lines of code with a single line and a new class. Now if we had the same scenario where we had to change our constructor function we would only had to change a few lines in our factory method.

Summary

In this post we went over the advantages of using the Factory Method patterns to simplify or code, and make it more manageable. Although the examples in this post where simple ones, the purpose was to exemplify the basic concept rather than a full explanation of every single usage scenario and advanced usages.

Please feel free to leave a comment with ideas, corrections or your opinion on the topic.

Playing with dependency injection in PHP

Mon, 06 Jan 2014 00:00:00 -0500

Dependency Injection is a software design pattern that allows avoiding hard-coding dependencies and makes possible to change the dependencies both at runtime and compile time.

By using Dependency Injection we can write more maintainable, testable, and modular code. All projects have dependencies. The larger the project the more dependencies is it bound to have; now having a great number of dependencies is nothing bad by itself however how those dependencies are managed and maintained is.

Dependency Injection is not a new pattern and it has been commonly used on many languages like Java, but this pattern is somewhat new in the PHP world and it’s gaining traction quickly thanks for frameworks like laravel

Let’s exemplify these concepts by creating a pair of classes first without dependency injection and then rewriting the code to use the dependency injection pattern; since I’m primarily a Magento developer I’ll be really original (wink, wink) and create a Product and a StockItem class.

At first glance the code looks pretty normal and it’s what many PHP developers would call good code, however if we take a closer look at it using the S.O.L.I.D principle we can see that the code actually has many problems:

The StockItem is tightly coupled with the Product class, and while this might not look bad on this particular example. Let’s imagine that we made a change to the StockItem class to include a new parameter, we would then have to modify every single class where the StockItem object was created.
The Product class knows too much, in this case the stock status and quantity, let’s say that our application can handle inventories from multiple sources and stores but for the same product. With that in mind it would be in our best interest to make the Product class know less about its inventory.
We just made our life harder when it comes to unit testing the code. Since we are instantiating the stockItem inside the constructor it would be impossible to unit test the Product class without also testing the StockItem class.

Let’s Inject something!

In the other hand by using dependency injection we can correct most of these problems, let’s take at the same code but using dependency injection:

Constructor Injection

By using dependency injection we have more maintainable code, in the previous example we are using a type of dependency injection normally referred as Constructor Injection. By doing a simple change we can reduce the level of complexity of our code and improve the overall quality; not to mention that now we can easily run unit tests.

Constructor injection is by far the most common method used and there are several advantages by using this particular type of injection:

If the dependency is required by the class and cannot work without it, by using constructor injection we guarantee that the required dependencies are present.
Since the constructor is only ever called when instantiating our object we can be sure that the dependency can’t be changed or altered during the object lifetime.

These two points make Constructor Injection extremely useful, however there is also a few drawbacks that make it unsuitable for all scenarios:

Since all dependencies are required, it’s not suitable when optional dependencies are needed.
While using class inheritance trying to extend and override the constructor becomes difficult.

Setter Injection

Another common type of dependency injection is called setter injection and the same code as above would look something like this:

As we can see, with Setter Injection the dependencies are provided to our class after it has been instantiated using setter methods. Setter Injection has a few advantages:

Allows for optional dependencies and the class can be created with default values.
Adding new dependencies is as easy as adding a new setter method and it won’t break any existing code.

Setter Injection might be more suitable for situations where more flexibility is required.

So is Dependency Injection right for my application?

At the end of the day is up to each developer the one that has to make the decision about what design patterns are the right fit for his application be it Dependency Injection or something else; that being said from personal experience using patterns like this might be an overkill for smaller projects.

But if you are working on large and long running project, then there is good chance that dependency injection might the right solution for your project.

The Transhumanist Revolution: Hubris

Wed, 01 Jan 2014 00:00:00 -0500

transhumanism (abbreviated as H+ or h+) is an international cultural and intellectual movement with an eventual goal of fundamentally transforming the human condition by developing and making widely available technologies to greatly enhance human intellectual, physical, and psychological capacities.

en.wikipedia.org/wiki/transhumanism

What is the transhumanist Revolution?

Or even better why is transhumanism a revolution? The word revolution evokes strong and most likely violent imagery; conflict and war are also typically associated with the word revolution; and I named this article that way for an opposite reason.

Is not that I want to associate violence and conflict with the transhumanist movement, quite the opposite; I believe that the transhumanist movement will only keep gaining strength and that it would be beneficial and in the best interest for humanity.

But that being said what better way of addressing the most negatives arguments against transhumanism than tackling the head on, this is the first on a series of articles where I attempt to address the main arguments and criticism against transhumanity.

Hubris

Overbearing pride or presumption; arrogance: “There is no safety in unlimited technological hubris” (McGeorge Bundy).

This argument basically accuses humanity of trying to play ‘God’. Now there are two branches on which the phrase ‘playing God’ is used one is in the secular sense, meaning that secular humanist believe that tampering with the human biology has higher risk than rewards and thus it should be avoided, and while the secular argument does have some validity; let’s go back to the first one and address the argument from the theological point of view.

Against ‘God’ plan

The Vatican has been one of the main critics of the transhumanist movement and any related technologies, going as far as to calling genetic manipulation an ‘immoral act against god’; this belief is derived from the notion that man was created in the image of God and that anything that changes the genetic identity of a man is radically immoral.

The problem is that the Vatican decided to draw the line of what defines a human being at the genetic code; but the fact of the matter is that defining what makes human is still very much out for debate. A hundred years ago a man with a prosthetic leg or even a hard would have probably raised the same concerns, or let’s even forget the artificial aspect for a second; the catholic church battled for many years on the subject of organ transplant.

In that sense there is a fair amount of hypocrisy going around when transhumanism is called playing ‘God’, if you truly want to make that argument from an honest position, you would have to reject every single advance that modern medicine has made:

Vaccines
Organ transplants
Artificial Organs
Cancer Treatments
Prosthetics (even the most basic ones)

And while modifying the genetic code and improving our own biology does sound like playing ‘God’; we have been playing ‘God’ for as long humanity has tried to cure disease and heal our sick.

The difference is that technology capable of taking us a step beyond treatment and prevention, to a point where human might be in control of their own evolution. And while is easy to be scared by change, we have to recognize that a big part of what define us has human beings is the constant search for change and improvement and not the stagnation that fear brings.

Technical Debt

Mon, 04 Nov 2013 00:00:00 -0500

Technical Debt, chances are that you as developer have heard that term at least once before; but ask yourself do you really understand technical debt and when is appropriate to use it.

The term “technical debt” was first coined by Ward Cunningham who compared technical complexity and debt in a 1992 experience report:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise. – Ward Cunningham (1992-03-26). “The WyCash Portfolio Management System”. Retrieved 2008-09-26.

Basically technical debt refers to the shortcuts or hacks that we do and that will have future repercussions on our application, technical debt can also be acquired by inaction, like not commenting code, not generating documentation, skipping testing, and so on.

So if you ever had to hack things to make it work or skip testing just to meet a deadline; then you incurred into technical debt without knowing. Technical debt is usually paid inform of time and money spend fixing problems that could’ve been easily avoided if proper practices would’ve been followed.

Technical debt is bad … right ?

So technical debt has to be bad since we are only generating problems for our future selfs correct? Well not exactly and this is a topic of much debate among developers, if properly managed technical debt can become and asset instead of problem.

And while the previous point is highly controversial, modern companies seem to be embracing technical debt as part of their “Standard” methodologies for development, specially does who follow Agile practices. So what does this mean, does it mean that technical debt is desirable even to some extend ?

Purposely incurring into technical debt implies sacrificing quality and injecting hacky code into our project, instinctively developers find the idea revolting (at least I do) however more often than not as developers we are forced to bend the rules.

If you’ve ever worked for an “Agile” development company or one that likes to work with extremely close deadlines; you know exactly what I mean with bend the rules.

And one can even incur into technical debt without being aware of it, so the truth is that no matter what you are likely to incur into some technical debt.

Dealing with Technical Debt

First rule of Dealing with Technical Debt: Never Ignore Technical Debt

The worst thing you can do with any kind of technical debt is ignored it, recognize that technical debt comes with the interest and sooner or later you will have to pay; even if you incurred into technical debt for good reasons is important that you get rid of it as soon as possible.

Second rule of Dealing with Technical Debt: Set a repayment plan

As we mentioned before debt is something that we want to get rid of as soon as possible, the same applies for technical debt.

For Technical Debt we want to specify a deadline to completely repay our debt back, this way you set a deadline and only allow the technical debt go on for a predetermined amount of time.

If we don’t pay attention, technical debt has the tendency to hang around, and the more time goes by the hard it becomes to remove a specific piece of technical debt.

Third rule of Dealing with Technical Debt: Set a debt ceiling

If you are not paying attention debt will pile on without you even noticing, for that reason is recommended that you setup a debt ceiling, a maximum amount that you can ‘borrow’ from your projects quality.

Setting a limit on how much debt a project can have helps keeps under control and forces you to think twice if that particular hack/change/compromise is really needed.

Conclusion

Sooner or later everyone is forced to acquire some level of technical debt, the previous rules will help you to it responsibly and you will better equipped to get rid of the technical debt faster.

That being said, while researching for this article I found that there was a fourth rule that was being overlooked and that it should be the golden rule when dealing with technical debt.

Golden rule of Dealing with Technical Debt: Avoid Technical Debt

This sounds kind of obvious but in practice is much harder than it sounds, and for most part is us, developers who are at fault. We need to be more vocal and push back to try to avoid Technical debt; when the Project Manager, Sales guy, client says “Just make it work”, “Hack it together, I don’t care “ or any similar cringe worthy phrases we need to stop and make them understand the consequences on the short and long term for the project and the code.

In the end the best way to deal with Technical Debt is not to have any to begin with.

We are consuming data the wrong way

Mon, 28 Oct 2013 00:00:00 -0400

Welcome to the information age where everything is connected, everything is in the cloud and where rivers of information flow directly to your screen. If you ever felt like you where drowning in information you are not alone, we consume information – or at least try to – from dozens or even hundreds of different channels.

As we move forward in the Age of Information we only increase the different sources of information that are available to us, yet it seems our data management and consumption skills remain surprisingly the same.

Currently most people are not really capable of keeping up with the barrage of information generated by social media, emails and the army of blogs and news readers that most people use but fail miserable at doing so.

The Dumb web

Currently the Web is dumb, and by dumb I don’t mean useless nor I’m about to tell you go offline – or spend time in the sun ugh — What I mean is that the web is literally dumb, there is no intelligent capabilities to the way we browse the web.

We are essential using the same technology as it was invented 30 years ago, when the few websites where just black text and a white background. Sure, we have css5, html5, javascript, server side languages, etc.

All those techs are great for processing and displaying data but really don’t help that much with data consumption. We still get a lot of noise and irrelevant content, let’s say for example I visit nytimes.com

[insert nytimes.com screenshot]

As you can see the page is overflowing with information, excerpts, headlines and articles lists, but really of vast that amount of information hat is presented to me I’m only interested on two or three articles across one or maybe 2 different categories.

Sure I could go to a category and narrow the results even further, but that is exactly my point, we shouldn’t have to do that the nytimes.com and the website in general should be smart enough to present a curated list of articles that match our tastes and preferences.

Why are we forced to hunt for information, we spend hours everyday trying to find gold nuggets in mountains of digital noise. Our daily lives are overflown with information and it’s only getting worse.

Moving towards the smart web

There are many challenges that still need to be solved before we can have a truly smart web, privacy being one of the most complex ones, sharing data with third party sites it is always a concern.

However there are some really interesting approaches currently under development that provide interesting approaches to the problem of data management.

One of my favourite ones is the open source project Huginn currently under development, the idea behind Huginn is to provide a framework for creating and managing intelligent agents that can help with our daily live tasks among them data consumption.

Stop Disrupting Everything

Sat, 14 Sep 2013 00:00:00 -0400

If you have been hanging around in HackerNews for the last few months chances are that you have seen at least one post similar to the following ones:

Disrupt Social Media
Help me Disrupt Health Care
Disrupting Mass Surveillance
Disruptive Innovation
Disrupt Sleep
Disrupt yourself

and even - Indian Inventor Disrupts The Period Industry

And the list goes on and on, disrupt is the new magical buzzword used by startups across the globe, and while I don’t have a problem with the word disrupt or even the general use of buzzwords to describe concepts; I do have a problem with the inherited mentality that comes with the use of such magical buzzwords.

I propose that the use of such buzzwords murks the message of the startups using them to describe their product or service and even worse can damage the efforts of startups that are not jumping into the disruption bandwagon.

But wait a minute that can’t be true, buzzwords are technical words or phrases that have become fashionable to explain certain terms within the industry, so they must be harmless well this is often not the case since these buzzwords come attached with some heavy baggage.

Let’s put things into context and define what disrupt actually means and how is being used:

1 a : to break apart : rupture b : to throw into disorder agitators trying to disrupt the meeting 2 : to interrupt the normal course or unity of definition from merria-webster.com

Let that definition sink for a second, disrupt means break apart and interrupt; by that definition a majority of the articles, startups and new techs have been called disruptive are NOT actually disruptive.

Not all innovation disruptive nor it should be, disruption – at least on terms of tech – means that a new technology comes to displace an existing one.

With that said, technologies and startups can be disruptive but it’s really difficult to assess a technology as disruptive until after they have actually had measurable impact on an industry.

So please stop calling yourself or your startup disruptive unless you actually understand what you are saying and specially if your startup is on it’s earlier stages or not even launched, otherwise you will only look foolish and subtract validity to your startup/idea/self.

5 Rules for Writing Great Commits

Thu, 05 Sep 2013 00:00:00 -0400

One of the most underestimated skills that a developer can have is the ability to make well documented, clean commits. Writing good commits can save everyone involved in project tremendous amounts of time, money and effort.

Other developers can quickly see what changes you made and why, and is extremely useful when tracking down when bugs where introduced.

A Skill that requires practice

Choose a project, preferably one that you have been working for some time and has had least a few dozens commit. Now take a look at your commits and try to back track your work. Not pretty right? At least it wasn’t for me; I had to take a hard look at my workflow and realize that I was approaching commit messages more than an annoyance that needed to be quickly bypassed instead using them as valuable tool for team and myself.

In order to help me make my commits consistent I setup the following 5 basic guidelines that my commit messages should try to implement:

1- Keep your commits clean

By clean we are referring at the actual files that are part of a commit; and only adding the files that really need to be part of the actual commit or even the repository.

I have see many times commits that include log files, tmp/x folders, IDE configuration files, media files. This is not only unnecessary but adds a lot of overhead and possibly increases the size of a repository.

2- Commit early, Commit often

Commit as soon as you have one change ready, thing of your commits as brief snapshots you can use to roll back. Or even better different save games that allow you to go back at specific spots and approach a problem different.

By committing often we are creating a development history that we can use as reference and in the worst case scenario to roll back in case of a problem.

3- One whole change per commit

While I was already doing this with my regular commits I found that many new developers where packing multiple changes into a single commit, and writing messages like:

Fixes vulnerability #222 Changes css class Rewrites controller

Having multiple changes all into a single commits makes a bug harder to track down and complicates the process of rolling back

4- Specify why was the change made

Let’s take a quick look at the previous message not only are multiple changes applied inside the same commit but we don’t have any of clue of what was the reason behind that change.

One new practice that applying is adding a small reason sentences with a description of why the change was implemented for example:

Rewrites controller Reason: The controller need be changed to implement x and y feature

5- Specify what was changed

Ok so our original commit message looks better but something still missing from it, we are not specifying that we change exactly, saying:

Rewrites controller

It’s not very specific a much better commit message would be:

Rewrites some/module/CheckoutController.php

This simple guidelines can save you of a lot of headaches down the road, and if another developer has to look back at the changes that where implemented on a project, he is able to quickly follow the history of the project.

Fitbit Flex and the Quantified Self Movement

Thu, 29 Aug 2013 00:00:00 -0400

I recently bought a Fitbit Flex, the Flex is a fitness tracker wristband, very similar to the Jawbone UP or the Nike Fuel band. However I have found the Flex to be extremely comfortable to use, accurate and very useful for keeping track of my daily fitness and health.

But why go through all the trouble of tracking my meals, and wearing a fitness tracker 24/7; well the reason is very simple I want to improve my general health and become a more effective person.

I believe that information empower us to make better healthier decisions, and this has certainly proven to be true in my case after a my first month of using the Fitbit Flex, I been able to lose 2KG just by changing my eating habits and doing a little more exercise every day.

Tracking what I eat and how much I exercise is only the beginning, I want to get to a point where I can track most if not everything of what happens on my daily life, looking for ways to do this I found about the Quantified Self movement.

The Quantified Self is a movement to incorporate technology into data acquisition on aspects of a person’s daily life in terms of inputs (e.g. food consumed, quality of surrounding air), states (e.g. mood, arousal, blood oxygen levels), and performance (mental and physical).

So without actually being aware of it I took my first step on the Quantified Self world, and started wondering what else could I possibly track and what I could do with that information.

After some consideration I want to keep tracking more of my biometric information before moving into things like the memoto or recording video of my waking hours.

Right now the Fitbit Flex tracks my activity and sleep quality, plus by using their dashboard I can track what food I eat and my caloric balance, which is great if you are trying to lose weight.

The Flex has also replaced my morning alarms since it comes with a neat silent alarm that wakes you up by vibrating.

Superficial tracking

While I love my FitBit Flex, It only tracks a fraction of the information that I hope to eventually track. For example the next vector of personal information that I want to start tracking is my heart rate and blood pressure.

Sure, I could take measurements daily and add that information to the Fitbit dashboard, but this sort of defeats the purpose of life logging, which in my opinion should be as friction less and possible and require minimum input from the user.

Plus I want to have a detailed log minute by minute of my heart rate and blood pressure, this information could later be used for correlating it with work related stress, activity and so on.

Another area that I feel could be improved is sleep tracking, the Flex tracks my ‘sleep efficiency’ but it does it based on movement so is not really that accurate. Ideally I want to track my REM patterns, temperature, heart rate, blood pressure, and so on while sleeping.

Conclusion

The Fitbit Flex is a great way to get started with the quantified self movement and get your feet wet without having to spend too much money or effort.

Even if you are just looking to lose some weight or track what you are eating, the Flex is a great choice.

Burnout

Fri, 02 Aug 2013 00:00:00 -0400

Burnout: A state of emotional and physical exhaustion caused by a prolonged period of stress and frustration; an inevitable corporate condition characterized by frequent displays of unprofessional behavior, a blithe refusal to do any work, and most important, a distinct aura of not giving a shit. – Urban Dictionary

We all been there, feeling tired, bored and even sick of our work; burnout is very common in the tech industry, probably more common that most people want to realize.

There are countless posts, articles and even books that talk about preventing and/or dealing with burnout; exercise, meditation, changing jobs, go in on vacation, changing your diet or taking a bunch of pills (vitamins). All of those have been suggested by the internet gurus, web doctors and startup celebrities.

But I’m not going to tell you to do any of those things because frankly if you start exercising, meditation or doing whatever it helps you to fight burnout while you are burnout then this new habit are not going to stick, or you probably won’t do them correctly and at then it will be a waste of time that will only another ‘failure’ to your life and contribute to that burnout feeling.

So how do you combat burnout, well here is the secret … you don’t, at least not until you understand it, so let’s jump into some definitions of what burnout is and what is not.

According to wikipedia:

Burnout is a psychological term that refers to long-term exhaustion and diminished interest in work.

Ok, so first two signs of burnout is feeling tired and not giving a shit, but how do we get at this point in the first place; burning out it’s a gradual process, you don’t wake up one morning and suddenly think “I’m tired of my life and no longer give a shit” –…well maybe you but that’s a different set of issues not burnout.–

Burnout starts to set in when that feeling of accomplishment and satisfaction starts to vanish from your daily lives –burnout not only affects our professional lives but our personal ones too, for example one can be emotionally burnout– this is very noticeable in our professional lives where the majority of people expects to achieve clear results every time.

It’s all related as how as a society we perceive success and accomplishments, we have a very clear and narrow definition of what success means and how is supposed to look like and when we don’t achieve those specific goals we start to feel as a failure, depressed, irritated and eventually burnout.

The most effective way of fighting burnout is to preventing it to ever happen, to do that we need to change the way we perceive success and how we measure achievements; even in failure there are reasons to be learned.

Burnout sets in when we stop learning, when we stop thinking, when we don’t realize that even if we are not achieving the results we were hoping, there is still much to learn.

So don’t fight burnout, just don’t let it set in.

Week 1: 500 Words a day

Tue, 30 Jul 2013 00:00:00 -0400

A week ago I decided to follow the example of the amazing Nathan Barry in order to try kick start this blog and work on a couple of book projects that I want to launch. The basic idea revolves around creating the habit of writing a specific amount of words every day no matter what.

It has been a week since I started and so far it has been both a rewarding and challenging process, this method is not new by any means whatsoever some people recommend doing 750 words per day others swear that 1000 words is the ideal amount.

There is even an on-line editor which sole focus is on writing 750 words per day, the site is called 750words.com – shocking right? – now it really all depends on what you are trying to say and the ability of the writer.

While 750words.com sounded like an easy way to get started, I didn’t felt comfortable writing what might be privates thoughts or at least sensitive information without any control or guarantees of how the information was stored and guarded.

For my daily goal I settled to start with 500 words; 500 words seemed to be an achievable goal that would keep me motivated, but challenging enough where I would actually had to put some effort.

Tricking Myself to Win

Setting a habit is hard, specially when is something that requires effort or makes uncomfortable (read exercise); writing is not different, you are not going to be on the top of your game each day and the infamous writers block my hit at any time and without warning.

So I took preventive methods to help me out and basically tricky myself pass any obstacles, the first thing that I’m doing is keeping record of how many days in a row I been keeping up with my goal, for tracking this I’m using and iPhone app called Commit by Nathan Barry.

The app is very simple and easy to use, you can set daily goals and if you want to a reminder; the app will ask you each day if you have met your goal. The more you keep the chain going the less likely is for you to break it; you also get instant feedback and a sense of achievement.

And thats exactly what I wanted, to feel that sense of achievement each day, that rush of satisfaction of completing a challenging task, basically I was trying to trick my brain into becoming addicting to writing.

We are what we repeatedly do. Excellence, then, is not an act, but a habit. - Aristotle

The first two days writing 500 daily words seemed easy enough, I was even exceeding the quote for a few hundreds words during the first days. By Thursday I was starting to slowdown.

Slowing down is not a bad thing unless you stop completely, I found that changing topics and working a different draft helped me to write pass my mental roadblocks and get back into a writing flow.

The result

After just my first week I feel incredibly productive and I have build a considerable buffer of posts, that I will be posting every four days or so in this blog.

Along with that I’m also making progress on my new book project (shhh! Don’t tell anyone).

First steps on HHVM

Sat, 27 Jul 2013 00:00:00 -0400

On a previous post Introduction to HHVM we went over HHVM’s history and the potential of running our PHP applications on top of it. Currently a few applications are fully supported like wordpress and drupal; more complex applications like Magento are still not 100% with HHVM due to bugs in the HHVM implementation.

The first thing that we need to in order to start developing with HHVM is to setup a proper environment, for this case we are going to use a Vagrant Box.

Skipping the Installation

If you want to skip the whole setup you can download the configured vagrant box by doing the following:

mkdir vagrant-hhvm
cd vagrant-hhvm
vagrant init
vagrant box add hhvmdev https://www.dropbox.com/s/5qyjkes49nk5abt/package.box

Installing Vagrant

Download and install the appropriate version of Vagrant for your machine.

Downloading Ubuntu Precise (12.04)

mkdir ~/dev/vagrant/
cd ~/dev/vagrant/
vagrant init

This will create a vagrant configuration file, this file is used for setting the configuration and provisioning our new vagrant box. We also need to download and ISO image of the Ubuntu 12.04 vagrant can do this with a single command.

vagrant box add base http://files.vagrantup.com/precise64.box

Once vagrant finishes downloading the Ubuntu image we can start our vagrant box by typing:

vagrant up

Vagrant runs in headless mode (meaning there is no screen or interface, everything runs in the background) in order to be able do anything with our new box we need to open a ssh session to it. If you are working in OSX or Linux you are luck, you can login into your vagrant box by typing:

vagrant ssh

Windows users can still run the command to get instructions on how to connect with vagrant by using putty or xshell.

Once we are logged it into our brand new vagrant box we are ready to start installing HHVM

Installing HHVM

Facebook currently provides binary distributions of the HHVM, but unfortunately this binary files are nowhere near the latest updates; so in order to have the latest bug fixes and patches we have to build HHVM from the repository HEAD

Setting up the build tools

We need to install some base tools before we can actually build HHVM:

sudo apt-get update
sudo apt-get install git-core cmake g++ libboost1.48-dev libmysqlclient-dev \
libxml2-dev libmcrypt-dev libicu-dev openssl build-essential binutils-dev \
libcap-dev libgd2-xpm-dev zlib1g-dev libtbb-dev libonig-dev libpcre3-dev \
autoconf libtool libcurl4-openssl-dev libboost-regex1.48-dev libboost-system1.48-dev \
libboost-program-options1.48-dev libboost-filesystem1.48-dev wget memcached \
libreadline-dev libncurses-dev libmemcached-dev libbz2-dev \
libc-client2007e-dev php5-mcrypt php5-imagick libgoogle-perftools-dev \
libcloog-ppl0 libelf-dev libdwarf-dev libunwind7-dev subversion

Building HHVM

Next, we need to get a copy of HHVM GitHub repository:

mkdir ~/hhvm-dev/
cd ~/hhvm-dev/
git clone https://github.com/facebook/hiphop-php

Before jumping ahead and compiling HHVM we need to setup some third-party libraries required to support some of the PHP language features:

libevent

git clone git://github.com/libevent/libevent.git
cd libevent
git checkout release-1.4.14b-stable
cat ../hiphop-php/hphp/third_party/libevent-1.4.14.fb-changes.diff | patch -p1
./autogen.sh
./configure --prefix=$CMAKE_PREFIX_PATH
sudo make && sudo make install

libCurl

git clone git://github.com/bagder/curl.git
cd curl
./buildconf
./configure --prefix=$CMAKE_PREFIX_PATH
sudo make && sudo make install

Google glog

svn checkout http://google-glog.googlecode.com/svn/trunk/ google-glog
cd google-glog
./configure --prefix=$CMAKE_PREFIX_PATH
sudo make && sudo make install

JEMalloc 3.0

wget http://www.canonware.com/download/jemalloc/jemalloc-3.0.0.tar.bz2
tar xjvf jemalloc-3.0.0.tar.bz2
cd jemalloc-3.0.0
./configure --prefix=$CMAKE_PREFIX_PATH
sudo make && sudo make install

Now we are ready for compiling HHVM:

cd hiphop-php
export CMAKE_PREFIX_PATH=`pwd`/..
export HPHP_HOME=`pwd`
cmake .
make

If the compiler ran successfully and we didn’t get any errors, we should be able to check HHVM version by running:

 hphp/hhvm/hhvm --version

Let’s create a symlink to make our lives a little easier:

sudo ln -s ~/hhvm-dev/hiphop-php/hphp/hhvm/hhvm /usr/bin/hhvm

Next steps

Let’s test HHVM by setting up a quick PHP script:

mkdir -p ~/dev/scripts/
vim ~/dev/scripts/hhvm-test.php

And copy the following code:

<?php echo "Hello world, I'm running HHVM\r\n"; ?>

Save the file, now in order to run or PHP applications through HHVM is important to understand how the differ from a normal LAMP stack(we will cover this on a later post), for now we can run our test script by running:

hhvm ~/dev/scripts/hhvm-test.php

If you see the message printed on your console means that the setup was successful and we are now running HHVM.

What’s next

In the next post I’ll cover the installation of Wordpress under HHVM and the basis for debugging an application running on HHVM.

An Introduction to HHVM

Wed, 24 Jul 2013 00:00:00 -0400

HHVM (HipHop Virtual Machine) converts PHP code into a high-level bytecode (commonly known as an intermediate language). This bytecode is then translated into x64 machine code dynamically at runtime by a just-in-time (JIT) compiler. In these respects, HHVM has similarities to virtual machines for other languages including C#/CLR and Java/JVM.

A little big of history

In early 2008 Facebook began working on HipHop(now HPHP), a PHP execution engine; its original motivation was to convert Facebook massive PHP code base into C++ in order to save resources and increase the application performance. The original release was known as HPHPc a PHP to C++ compiler.

For next 2 years Facebook continued working on HipHop adding HPHPi ( a ‘developer mode’ version of HPHP) and HipHop debugger known as HPHPd, this allowed developers to watch and step through the code and interactively debug PHP applications running on the HipHop platform.

At it’s peak, HipHop PHP code showed up to 6x times better performance than its ZEND counterpart. However, there where several drawbacks to this first iteration of HipHop:

HPHPc did not fully supported the PHP language, most noticeably the lack of support for create_function() and eval() –Honestly, I don’t see the lack of support for the last one as something bad– functions.
Facebook developers had to maintain two difference engines (HPHPc and HPHPi) with resulted in duplication of efforts and waste of resources.
Finally, HPHPc required a vastly different deployment process which will hurt adoption by PHP developers.

In light of these problems Facebook took two key actions on early 2010, the first was to open source the HipHop platform. – Open sourcing a project like this is a great way to build a community around the project and getting external help from that community –/

At the same time Facebook started the development of the modern version of HipHop, known as HHVM (HipHop Virtual Machine), HHVM improves the strengths of HPHPc and corrects many of the critical problems.

HHVM was build on top of HPHPc and it works by converting PHP code into high level bytecode(an intermediate language) this bytecode is the translated into machine code dynamically at runtime by a JIT (Just-In-Time) compiler.

If you are like me you probably have a vague recollection of the concepts of bytecode, machine code and Just-In-Time compilers, so let’s take a momentarily side step and quickly review these concepts and how the they play a key role on HHVM.

Bytecode, Machine code, JIT, Oh my …!

Bytecode: Is a non human readable code designed for efficient execution by an interpreter or compiler. When HHVM first loads our project it converts all the PHP code into this intermediary form of bytecode; the bytecode generate is not particular to any type of architecture and could be portable among different systems.
Machine code: Are a set of instructions designed to be executed directly by the CPU. If you ever played with Assembly (Who hasn’t !?) you know very well what machine code is, for those very few of you who haven’t had the pleasure of working on assembly or similar languages; machine code is generated by the compiler or interpreter and then handed to the CPU
JIT (Just In Time) compiler: Just in time compilation is a technique used to improve performance of software, this is achieved by compiling bytecode during execution, the byte code is store in memory and then the JIT compiler loads and compiles as many sections of the byte code as needed.

The performance and speed gained by applying these techniques is what gives HipHop and subsequently HHVM its core strengths. Keeping a PHP code base while achieving performance comparable to compiled applications.

Currently HHVM supports PHP 5.4 almost on its entirety, however there are still numerous bugs that prevent some applications from running, for that reason Facebook has set as goal to have the top 20 open source PHP applications running on HHVM. The first popular application to achieve this was Wordpress.

What’s next

So now that we have a better understanding of what HHVM does and the advantages if running it we can start testing our applications on HHVM. In a subsequent post I will be covering the setup of a dedicated Vhost for HHVM, running benchmarks against your applications and eventually (fingers crossed) how to run Magento on HHVM.

More than human

Sun, 21 Jul 2013 00:00:00 -0400

In the last couple years, we have seen the emergence of new and exciting technologies aimed towards closing the gap between man and machine; projects like Google glass, scanadu the real life ‘tricorder’ and the myriad of fitness bands are only a few examples of how not only technology has become part of daily lives but is slowly becoming part of what we are and how we define our identity.

Transhumanism is an international intellectual and cultural movement supporting the use of science and technology to improve the human condition…

en.wikipedia.org/wiki/Transhumanism

From a technologist point of view, this phenomenon is really interesting, but I find even more interesting the social implications of this line of technological development; which is what I intent do explore on this particular article.

I’m a huge fan of science fiction and trans-humanism, that been said I’ve found that many trans-humanists and futurologist often fail to factor-in socio-economic and political variables in their predictions, and in that sense they might end been to optimistic on the time lines that they propose.

Ray Kurzweil, father of the singularity movement and probably the most famous futurologist is a clear example of over optimistic predictions while he is more than likely correct on his technological predictions he fails to factor the possible blow-back from political groups, religious institutions and the resistance that is common with change.

Google Glass, which we could call the earlier beginning of trans-humanist technologies is already getting resistance well before of its public release and efforts have started in the US, and UK to get it banned.

What will happen when more esoteric technologies are made available to us, what kind of resistance we can expect when the first humans can and start modifying their own biology.

The problem with religion

What would happen if as Kurtweil expects, we get the first Immortal humans in the next few decades? How will a discovery like this affect the world religions, who base their core dogmas in the beliefs of an afterlife, and the rewards (or punishment) that come with it?

If death is no longer part of the human condition and becomes an avoidable, even reversible process; how would the religious leaders react?

If such evens happened tomorrow and this treatment was widely available, then is very likely that we would see even larger acts of terrorism committed the religious extremist of the world; a Jihad on science, Christian extremist bombing hospitals and carrying ‘Gods will’.

The Future of Humankind

Regardless of the singularity and the trans-humanism movement, humankind needs to mature; collectively we are in our teenagers still trying to figure out whom we are, and we want to do.

Unfortunately, the stakes are much higher, and if we do not get our act together as species there is a real chance that we will not make it see the promises of the singularity.

Blog relaunched using jekyll

Mon, 01 Jul 2013 00:00:00 -0400

Over the last 2 years I have tried several times to kickstart coderoncode.com, and a long with it I have tried multiple blogging platforms (wordpress, obtvse, etc) I even tried coding my own blogging solution; all without much success.

From each of these attempts I have learned valueable lessons, so far I have spent more time tweaking the platform and the style than actually writing; so if I’m to succeed this time I need a blogging platform that actually lets me write and doesn’t get in the way.

Meet Jekyll

Jekyll is a simple, blog-aware, static site generator. It takes a template directory containing raw text files in various formats, runs it through Markdown (or Textile) and Liquid converters, and spits out a complete, ready-to-publish static website suitable for serving with your favorite web server.

As content generators go, Jekyll is truly amazing, while I don’t (yet) intent to list the full set of features or how to use it we can over the main features of Jekyll

Clear directory structure and scaffolding generation.
Support for assets and downloads.
Preview server.
Configurable in YAML.
Supports drafts and future publications dates.

About

Allan MacGregor is a developer living and working in Toronto. Originally from Mexico, he freelanced his way through college before moving to Canada, in 2009. While he designed and developed dozens of projects with clients large and small, specializing in ecommerce websites; during that time he was one of the early adopters of Magento. In early 2010, he moved to Toronto to work at Demac Media as Magento Lead Developer.

Since joining Demac, Allan has worked on numerous projects including Integrations with CanadaPost, Optimal Payments and, Masterpass, several internal tools, and dozens of Magento Stores.

My Books

Released:

What I’m working on

Some fun projects I been working on:

MageVagrant GitHub
MageHack-Voting GitHub
Magento Whoops GitHub
Magento Xhprof Integration GitHub

For comments or questions, feel free to ask me on Twitter or shoot me an email at info at allanmacgregor dot com.

Books

Test Driven Magento Extension Development

Learn how to effectively build and test extensions using TDD/BDD and tools like PHPspec, Behat and PHPUnit.

Magento PHP Developer’s Guide

Get started with the flexible and powerful e-commerce framework, Magento.