One of the awesome things about my last job is that I got to open source my work. More companies should do this! Among the benefits of open-sourcing work is it presents evidence of previous work and experience for the employee and for the employer it means projects get exposure and contributions from outside.
My main project is a SOP/experiment definition management system. It was written in PHP using the CodeIgniter framework with a little Zend Framework and the data is stored in a MySQL database. I'm close to finishing the paper for the project. Check out the code in the meantime: https://github.com/mpi2/impress
For the main project I created an OWL Ontology using a combination of Protégé and the Java OWLAPI to generate ontology classes and individuals from the items in the database. For those of you who want to figure out how to use the OWLAPI to build ontologies I am sure the code will help you figure it out: https://github.com/mpi2/impress_owl
Also, I'd like to point out another project I wrote which is mappings of core XML datatypes into PHP classes (with validation checking): https://github.com/mpi2/xmldatatype.
And a reminder to a previously mentioned project, PhpObo, which is a OBO file format parser and builder written in PHP: https://github.com/mpi2/PhpObo
Saturday, 7 June 2014
Friday, 18 April 2014
Permutations
The other day a friend was seeking advice about passwords and password length and such and I just told him the strength of a password is based on however many characters (n) he has available to him to the power of the length of the string (l), n^l, and that he should also use non-alphanumeric characters as well and make sure it's long enough and memorable to him. Using the formula, a 4 letter password using just digits is 10^4, or 10000 possible combinations.
That got me thinking about cracking and finding combinations of passwords if you know which characters were used and in what frequency. Basically you would need to jumble the characters up and try the different combinations but because you know which characters were used the entropy is far less, l^2 - l. This would mean a 4 letter passwords using the characters abcd would have 4^2 - 4, or 12 possible combinations.
I had to put a little thought into how I was going to code this. The first thing you do when you have to write an algorithm is figure out how you would do it in real life and I thought up the idea of swivelling characters round. So, if we have the starting characters abcd, you would lock the first character and swivel the other three round:
abcd
adbc
acdb
Then, you would shift the first character (a) off the original array and push it onto the end, lock the new first character (b), and do the swivelling of the other three again:
bcda
bacd
bdac
and so on. So when I first tried to solve this problem I did it the easy way by using the php array functions:
<?php
/**
* Produce permutations of a string using php array functions
* @author Ahmad Retha
* @license public domain
*/
$s = "abcd"; //starting letters
$a = str_split($s); //converted to char array
$l = count($a); //length of array
$i = 0; //initialize counter for loop
$r = array(); //result array holds the combinations
while ($i < $l) {
$b = array_slice($a, 1); //new array missing first element
$j = 0; //initialize counter for inter-swivel
while ($j < $l - 1) {
array_push($b, array_shift($b)); //swivel smaller array
//store result in $r
$r[] = implode('', array_merge((array)$a[0], $b));
$j++;
}
array_push($a, array_shift($a)); //swivel initial character
$i++;
}
?>
But I was not satisfied with this and set about rewriting it in a more efficient way using array index manipulation to move elements around. This code is a little longer but more efficient and runs a tad faster:
<?php
/**
* Produce permutations of a string through array manipulation
* @author Ahmad Retha
* @license public domain
*/
$s = "abcd"; //starting letters
$a = str_split($s); //converted to char array
$l = count($a); //length of array
$r = array(); //result array holds the combinations
$c = 0; //counter
$t = null; //temporarily holds first char
while ($c < $l) {
$t = $a[0]; //store first char
$n = 1; //initialize counter for inner loop
for ($i = 1; $i < $l; $i++) {
while ($n < $l) {
//temporarily store first char of inner substring
$u = $a[1];
//shift chars along in substring
for ($j = 2; $j < $l; $j++) {
$a[$j - 1] = $a[$j];
}
$a[$l - 1] = $u; //push substring temp char to end
$r[] = implode('', $a); //store the permutation
$n++;
}
//shift chars along in whole string
$a[$i - 1] = $a[$i];
}
$a[$l - 1] = $t; //push temp char to end
$c++;
}
?>
This is sample output of the 12 combinations held in $r:
acdb
adbc
abcd
bdac
bacd
bcda
cabd
cbda
cdab
dbca
dcab
dabc
That got me thinking about cracking and finding combinations of passwords if you know which characters were used and in what frequency. Basically you would need to jumble the characters up and try the different combinations but because you know which characters were used the entropy is far less, l^2 - l. This would mean a 4 letter passwords using the characters abcd would have 4^2 - 4, or 12 possible combinations.
I had to put a little thought into how I was going to code this. The first thing you do when you have to write an algorithm is figure out how you would do it in real life and I thought up the idea of swivelling characters round. So, if we have the starting characters abcd, you would lock the first character and swivel the other three round:
abcd
adbc
acdb
Then, you would shift the first character (a) off the original array and push it onto the end, lock the new first character (b), and do the swivelling of the other three again:
bcda
bacd
bdac
and so on. So when I first tried to solve this problem I did it the easy way by using the php array functions:
<?php
/**
* Produce permutations of a string using php array functions
* @author Ahmad Retha
* @license public domain
*/
$s = "abcd"; //starting letters
$a = str_split($s); //converted to char array
$l = count($a); //length of array
$i = 0; //initialize counter for loop
$r = array(); //result array holds the combinations
while ($i < $l) {
$b = array_slice($a, 1); //new array missing first element
$j = 0; //initialize counter for inter-swivel
while ($j < $l - 1) {
array_push($b, array_shift($b)); //swivel smaller array
//store result in $r
$r[] = implode('', array_merge((array)$a[0], $b));
$j++;
}
array_push($a, array_shift($a)); //swivel initial character
$i++;
}
?>
But I was not satisfied with this and set about rewriting it in a more efficient way using array index manipulation to move elements around. This code is a little longer but more efficient and runs a tad faster:
<?php
/**
* Produce permutations of a string through array manipulation
* @author Ahmad Retha
* @license public domain
*/
$s = "abcd"; //starting letters
$a = str_split($s); //converted to char array
$l = count($a); //length of array
$r = array(); //result array holds the combinations
$c = 0; //counter
$t = null; //temporarily holds first char
while ($c < $l) {
$t = $a[0]; //store first char
$n = 1; //initialize counter for inner loop
for ($i = 1; $i < $l; $i++) {
while ($n < $l) {
//temporarily store first char of inner substring
$u = $a[1];
//shift chars along in substring
for ($j = 2; $j < $l; $j++) {
$a[$j - 1] = $a[$j];
}
$a[$l - 1] = $u; //push substring temp char to end
$r[] = implode('', $a); //store the permutation
$n++;
}
//shift chars along in whole string
$a[$i - 1] = $a[$i];
}
$a[$l - 1] = $t; //push temp char to end
$c++;
}
?>
This is sample output of the 12 combinations held in $r:
acdb
adbc
abcd
bdac
bacd
bcda
cabd
cbda
cdab
dbca
dcab
dabc
Tuesday, 14 January 2014
Installing PEAR packages through Composer
Did you know that you can obtain PEAR packages via Composer? And you don't need to install or set up PEAR or configure global settings!
A work colleague was talking about difficulties setting up CPAN for Perl on a server and that got me thinking about dependency managers and repositories and the options available in the PHP world. The modern way to manage dependencies and find libraries/packages is using Composer and Packagist but the old way was through using PEAR which was a global dependency manager with a repository of useful packages. I started to wonder if it was possible to load an old-style PEAR package through the new Composer dependency manager and after some fiddling about I got it working successfully.
Not long ago the PEAR guys decided to create a new version of PEAR so there are now two PEAR repositories available: classic PEAR (http://pear.php.net) and PEAR2 (http://pear2.php.net) which is also known as Pyrus. Most of the packages are still stuck in the old PEAR repository. So let's pick a test package from each of them to install. For PEAR2 I'm going to try the package suggested in the Composer documentation, PEAR2_HTTP_Request, and for old PEAR I'm going to try Numbers_Roman.
Open up the composer.json file and edit it so it looks something like this:
{
"repositories": [
{
"type": "pear",
"url": "http://pear.php.net"
},
{
"type": "pear",
"url": "http://pear2.php.net"
}
],
"require": {
"pear-pear2/PEAR2_HTTP_Request": "*",
"pear-pear/Numbers_Roman": "*"
}
}
The two different PEAR repositories are defined in the repositories section of the composer.json file. Note the first two words (pear-pear and pear-pear2) used in the paths of the require values. This allows Composer to know which repository to look in. Details about choosing alternative channels are in the Composer documentation.
Then run php composer.phar update to install the new packages.
Now we can create a new PHP file to test the packages just installed. Here's my test code. Note that PEAR2 packages are namespaced, hence the use command:
<?php
require 'vendor/autoload.php';
use pear2\HTTP\Request,
pear2\HTTP\Request\Adapter\Curl;
$url = 'http://www.yahoo.com/';
$request = new Request($url, new Curl());
$response = $request->sendRequest();
$nr = new Numbers_Roman();
//http response code 200 or 'CC' in roman numerals means success
if ('CC' == $nr->toNumeral($response->code)) {
echo "Successfully accessed $url";
} else {
echo "An error (HTTP{$response->code}) occured while trying to access $url";
}
?>
And that's it. Composer is just awesome!
A work colleague was talking about difficulties setting up CPAN for Perl on a server and that got me thinking about dependency managers and repositories and the options available in the PHP world. The modern way to manage dependencies and find libraries/packages is using Composer and Packagist but the old way was through using PEAR which was a global dependency manager with a repository of useful packages. I started to wonder if it was possible to load an old-style PEAR package through the new Composer dependency manager and after some fiddling about I got it working successfully.
Not long ago the PEAR guys decided to create a new version of PEAR so there are now two PEAR repositories available: classic PEAR (http://pear.php.net) and PEAR2 (http://pear2.php.net) which is also known as Pyrus. Most of the packages are still stuck in the old PEAR repository. So let's pick a test package from each of them to install. For PEAR2 I'm going to try the package suggested in the Composer documentation, PEAR2_HTTP_Request, and for old PEAR I'm going to try Numbers_Roman.
Open up the composer.json file and edit it so it looks something like this:
{
"repositories": [
{
"type": "pear",
"url": "http://pear.php.net"
},
{
"type": "pear",
"url": "http://pear2.php.net"
}
],
"require": {
"pear-pear2/PEAR2_HTTP_Request": "*",
"pear-pear/Numbers_Roman": "*"
}
}
The two different PEAR repositories are defined in the repositories section of the composer.json file. Note the first two words (pear-pear and pear-pear2) used in the paths of the require values. This allows Composer to know which repository to look in. Details about choosing alternative channels are in the Composer documentation.
Then run php composer.phar update to install the new packages.
Now we can create a new PHP file to test the packages just installed. Here's my test code. Note that PEAR2 packages are namespaced, hence the use command:
<?php
require 'vendor/autoload.php';
use pear2\HTTP\Request,
pear2\HTTP\Request\Adapter\Curl;
$url = 'http://www.yahoo.com/';
$request = new Request($url, new Curl());
$response = $request->sendRequest();
$nr = new Numbers_Roman();
//http response code 200 or 'CC' in roman numerals means success
if ('CC' == $nr->toNumeral($response->code)) {
echo "Successfully accessed $url";
} else {
echo "An error (HTTP{$response->code}) occured while trying to access $url";
}
?>
And that's it. Composer is just awesome!
Saturday, 4 January 2014
Reading OBO Files in PHP using PhpObo
OBO files are specially formatted text-based human-readable files that contain ontologies, terms and descriptions, that describe a domain. At work I had a requirement to look through and double check the status of certain terms in the Mammalian Phenotype (MP) ontology. My first approach was to loop through the XML-based OWL format of the MP ontology, but I soon realised the OWL file was infrequently updated and I needed something much more current to work with. The other thing I noted was that other ontologies were not available in OWL format either, so what was really needed was a way to scan through the OBO files.
I had a little search and saw that there were solutions in Java and a Perl library but nothing in PHP, which is what my main application is written in so I decided to write my own OBO parser in PHP. Initially, it was going to be a really simple script to just loop through the OBO file but after reading the OBO format specification I realised I might as well write a proper library and set about writing PhpObo for myself and anyone else who would need it.
I have published the PhpObo library on Github under the Apache 2.0 license so feel free to use it, modify it and contribute if you wish. It is written in a flexible object oriented manner so you can swap out virtually any class from the library with your own version or extend its functionality. It is not a complete solution since it only works with one document at a time and doesn't resolve external ontology dependencies. But it does serve most people's needs and allow you to loop through any OBO file and it also allows you to generate your own OBO document using either an OOP or an Array-based (ArrayAccess) approach and serialize it out in the OBO file format.
If you wish to use PhpObo in your PHP 5.3+ project, I recommend you use a PHP PSR-0 dependency manager and autoloader like Composer to import the PhpObo project via Packagist.
I had a little search and saw that there were solutions in Java and a Perl library but nothing in PHP, which is what my main application is written in so I decided to write my own OBO parser in PHP. Initially, it was going to be a really simple script to just loop through the OBO file but after reading the OBO format specification I realised I might as well write a proper library and set about writing PhpObo for myself and anyone else who would need it.
I have published the PhpObo library on Github under the Apache 2.0 license so feel free to use it, modify it and contribute if you wish. It is written in a flexible object oriented manner so you can swap out virtually any class from the library with your own version or extend its functionality. It is not a complete solution since it only works with one document at a time and doesn't resolve external ontology dependencies. But it does serve most people's needs and allow you to loop through any OBO file and it also allows you to generate your own OBO document using either an OOP or an Array-based (ArrayAccess) approach and serialize it out in the OBO file format.
If you wish to use PhpObo in your PHP 5.3+ project, I recommend you use a PHP PSR-0 dependency manager and autoloader like Composer to import the PhpObo project via Packagist.
Thursday, 5 December 2013
Spreading the Fear: A potential new form of information transmission is discovered
Recently a report was published in the journal Nature Neuroscience about an experiment where mice were taught to associate pain with a scent, and then the fear of this scent was passed on to their offspring and their offspring's litter without them having experienced the pain of the parent or grand-parent themself. If what this research indicates is true, then it could open up the flood gates to a new dimension of genetic research.
The key point to take from this research is that traits can be passed from parent to child and grandchild without there being an alteration of DNA! So there's another form of inheritance! This brings in to question one of the central dogmas in genetics that DNA is the only genetic material (in all species save for a few microbes). If this research is confirmed and an alternative mechanism for information transmission is defined, then it will open up a whole new way of looking at genetics and inheritance of traits, both physical and psychological and also various diseases.
This might be the first time such an idea has been proved without their being confounding genetic and environmental factors that might affect the results and there is evidence of physical changes in the brain too. But before we jump up and down like excited children, we need to be careful and see if the result can be reproduced in other mice strains and in other species. I expect they will.
Here's the article by Nature News: Fearful memories haunt mouse descendants
And the original research papers (pay-walled): Implications of memory modulation for post-traumatic stress and fear disorders by Ryan G Parsons and Kerry J Ressler in Nature Neuroscience 16, 146-153 (2013) doi:10.1038/nn.3296
The key point to take from this research is that traits can be passed from parent to child and grandchild without there being an alteration of DNA! So there's another form of inheritance! This brings in to question one of the central dogmas in genetics that DNA is the only genetic material (in all species save for a few microbes). If this research is confirmed and an alternative mechanism for information transmission is defined, then it will open up a whole new way of looking at genetics and inheritance of traits, both physical and psychological and also various diseases.
This might be the first time such an idea has been proved without their being confounding genetic and environmental factors that might affect the results and there is evidence of physical changes in the brain too. But before we jump up and down like excited children, we need to be careful and see if the result can be reproduced in other mice strains and in other species. I expect they will.
Here's the article by Nature News: Fearful memories haunt mouse descendants
And the original research papers (pay-walled): Implications of memory modulation for post-traumatic stress and fear disorders by Ryan G Parsons and Kerry J Ressler in Nature Neuroscience 16, 146-153 (2013) doi:10.1038/nn.3296
Wednesday, 30 October 2013
Javascript snippet to HTML encode foreign characters
This Javascript snippet is useful for converting characters in a language like Arabic to HTML Encoding for putting on web pages.
var a = "السلام عليكم"; //Arabic for Peace Be Upon You (Hello)
var h = a.replace(/(.)/g, function($1){
return ($1 == " ") ? $1 : "&#" + $1.charCodeAt() + ";";
});
The variable h now holds السلام عليكم
var a = "السلام عليكم"; //Arabic for Peace Be Upon You (Hello)
var h = a.replace(/(.)/g, function($1){
return ($1 == " ") ? $1 : "&#" + $1.charCodeAt() + ";";
});
The variable h now holds السلام عليكم
a.replace(/(.)/g, function($1){
if ($1 == " ")
return " ";
else
return "&#" + $1.charCodeAt() + ";";
})
"السلام عليكم"
"السلام عليكم"
"السلام عليكم"
"السلام عليكم"
console.log("السلام عليكم".replace(/(.)/g, function($1){
if ($1 == " ")
return " ";
else
return "&#" + $1.charCodeAt() + ";";
}));
Monday, 28 October 2013
Drupal 7: Referencing Views from inside Panel Pages
Example Scenario: I have created a Panel Page (e.g. Cars) and I want to
display a View list of Parts associated with any Car loaded through that Panel
page. Each Car node contains a reference (Entity Reference) field (field_parts_ref) to all the parts associated with it.
First, create your View, choose which fields to display, and add a context - select Content: Nid of the Car and on the next page you select Provide default value from the drop down list, select PHP Code from the drop down list, and add the following code:
First, create your View, choose which fields to display, and add a context - select Content: Nid of the Car and on the next page you select Provide default value from the drop down list, select PHP Code from the drop down list, and add the following code:
//get node id of parent passed from the panel and load the node into memory
$nid = $argument->view->args[1];
$node = node_load($nid);
//read the field which has references to the node id's of the child items and add them to $refs
$refs = array();
foreach((array)$node->field_parts_ref['und'] as $t){
if ( ! empty($t['target_id']))
$refs[] = $t['target_id'];
}
//return the array of child item references to the view in a comma-seperated list so it displays only those items
return implode(',', $refs);
Now scroll to the bottom and expand More and check Allow multiple values.
$nid = $argument->view->args[1];
$node = node_load($nid);
//read the field which has references to the node id's of the child items and add them to $refs
$refs = array();
foreach((array)$node->field_parts_ref['und'] as $t){
if ( ! empty($t['target_id']))
$refs[] = $t['target_id'];
}
//return the array of child item references to the view in a comma-seperated list so it displays only those items
return implode(',', $refs);
Now scroll to the bottom and expand More and check Allow multiple values.
Now you can create your Panel Page. Add your view as a content, but when it shows the interface to configure the View you need to select the Cars ID field in the Node Being Viewed section of the drop down, and make sure the Send arguments check box is checked.
That's it. It was a little tricky to figure this stuff out but I got there in the end, so I hope this gives some clue to anyone trying to get something working.
Tuesday, 1 October 2013
Things that annoy me about OOP in PHP
I've been doing quite a bit of "proper" OOP coding in PHP lately and made a note of common pitfalls I came across. This list is not a complete run-down of what's wrong with OOP PHP. To be honest, save for these pitfalls, proper coding of PHP is actually fun.
1. Having to import every single class you use manually
In Java you say import java.util.*; where the asterisk means everything in that module is available for use in this namespace. Easy to see what's going on and a real timesaver. While using standard PSR-0 autoloading conventions, PHP doesn't have library importing so for every class you use you have to import it or give the full classpath everytime you need to use it. You will often see PHP code with long lists of 'use' statements at the beginning of the file like so:
use mylib\app\util\LoremIpsumGenerator,
mylib\app\config\ConfigLoader,
anotherlib\tastyphp\orm\TastyORM,
randomlib\randomhouse\RealRandom;
2. SPL classes have to be referenced with a preceding backslash
Instead of the PHP engine doing a simple search to find a class, first in the namespace of the file, then in the Standard PHP Library namespace, you have to delimit an SPL class with a backslash every time you use it. Why can't I just say "new Exception()" instead of "new \Exception()"? Why do I have to use a backslash everywhere I use an SPL class? It's totally unnecessary and becomes annoying to code and irritating to debug
/**
* Yes, yes, I know I should use instanceof
* but I'm showing another SPL class being used
* @throws \Exception
* @param \Exception|mylib\app\exceptions\MyBaseException
* $exception
*/
public function captureMyExceptionThrowPHPException(\Exception $exception)
{
$rc = new \ReflectionClass($exception);
if ($rc->isSubclassOf('mylib\app\exceptions\MyBaseException')) throw new \Exception('My Exception: ' . $exception->getMessage());
else throw $exception;
}
3. Lack of object casting
Seriously, why can't I cast one type of object into another, even when the subject is a subclass of the object? I wish it was possible to do this: $e = (\Exception) new mylib\app\exceptions\MyBaseException('My exception'); That would save me a bit of typing and potential buggy code.
Java allows you to upcast an object, meaning a subclass can be easily converted into a parent class, but it is not possible to downcast a parent item into a subclass type. Ideally it should be able to do that but at least it is able to upcast. PHP can't cast anything except it's primitives and two complex types (int|string|float|bool|array|object).
4. Lack of proper documentation for SPL Classes on the PHP.net site
Common SPL Functions are well documented with community supplied code snippets for everything you can imagine so it's easy to copy and paste and you're done. PHP5 SPL Classes are poorly documented, which is a real shame because some crap and procedural code from the PHP4 days has now been re-implemented in a much cleaner, better and faster object oriented way. But you wouldn't know that because of the lack of information in the manual. More work needs to be put into this. More comments with usage examples are necessary. These are provided by the community so please contribute to the project with a comment or an example if you have anything helpful.
1. Having to import every single class you use manually
In Java you say import java.util.*; where the asterisk means everything in that module is available for use in this namespace. Easy to see what's going on and a real timesaver. While using standard PSR-0 autoloading conventions, PHP doesn't have library importing so for every class you use you have to import it or give the full classpath everytime you need to use it. You will often see PHP code with long lists of 'use' statements at the beginning of the file like so:
use mylib\app\util\LoremIpsumGenerator,
mylib\app\config\ConfigLoader,
anotherlib\tastyphp\orm\TastyORM,
randomlib\randomhouse\RealRandom;
2. SPL classes have to be referenced with a preceding backslash
Instead of the PHP engine doing a simple search to find a class, first in the namespace of the file, then in the Standard PHP Library namespace, you have to delimit an SPL class with a backslash every time you use it. Why can't I just say "new Exception()" instead of "new \Exception()"? Why do I have to use a backslash everywhere I use an SPL class? It's totally unnecessary and becomes annoying to code and irritating to debug
/**
* Yes, yes, I know I should use instanceof
* but I'm showing another SPL class being used
* @throws \Exception
* @param \Exception|mylib\app\exceptions\MyBaseException
* $exception
*/
public function captureMyExceptionThrowPHPException(\Exception $exception)
{
$rc = new \ReflectionClass($exception);
if ($rc->isSubclassOf('mylib\app\exceptions\MyBaseException')) throw new \Exception('My Exception: ' . $exception->getMessage());
else throw $exception;
}
3. Lack of object casting
Seriously, why can't I cast one type of object into another, even when the subject is a subclass of the object? I wish it was possible to do this: $e = (\Exception) new mylib\app\exceptions\MyBaseException('My exception'); That would save me a bit of typing and potential buggy code.
Java allows you to upcast an object, meaning a subclass can be easily converted into a parent class, but it is not possible to downcast a parent item into a subclass type. Ideally it should be able to do that but at least it is able to upcast. PHP can't cast anything except it's primitives and two complex types (int|string|float|bool|array|object).
4. Lack of proper documentation for SPL Classes on the PHP.net site
Common SPL Functions are well documented with community supplied code snippets for everything you can imagine so it's easy to copy and paste and you're done. PHP5 SPL Classes are poorly documented, which is a real shame because some crap and procedural code from the PHP4 days has now been re-implemented in a much cleaner, better and faster object oriented way. But you wouldn't know that because of the lack of information in the manual. More work needs to be put into this. More comments with usage examples are necessary. These are provided by the community so please contribute to the project with a comment or an example if you have anything helpful.
Sunday, 23 June 2013
Tortoiseshell-colored cats are usually female
I was reading something about cat behaviour and that reminded me of a fact that I learnt a long time ago, that almost all Tortoiseshell-coloured (black and orange patch) cats are female because of an X-linked gene. I googled it to remind myself of the genetics behind it and found an excellent page that explains it in detail: The genetics of Calico cats
Subscribe to:
Posts (Atom)