Ship it with Docker.. Tips!

Docker is fairly new, easy to learn with lots of potential to replace so many heavy weight processes from as small as Dev machines, QA automation, Continuous integration to Production deployment and complex infrastructure, Something like AWS CloudFormation that allow you to have your infrastructure scripted and versioned so it would be easy for developers or new comers to understand and adapt quickly to it! however AWS require you to have an account and sometimes services not for free, on the other side Docker can give you that! You could replicate your big infrastructure with versioning and configuration etc…. at every dev machine at no expense or per-requisites.

Great technology that will definitely change the cloud computing development road-map!
Docker run on Linux systems, also it has MAC and Windows support via Boot2Docker, very lightweight Linux VM just to run docker.


Mounting Volumes/Folders:

Using docker on Linux is less painful than using is on Mac and Windows in terms of volume mounting and shared folders, In linux you don’t have problem you can mount in folder in the Docker host (linux) to any docker container, but in MAC/Windows you need to place the folder you need to mount it in docker container in on of those places :

  • Users share at /Users
  • /Users share at /Users
  • c/Users share at /c/Users
  • /c/Users share at /c/Users
  • c:/Users share at /c/Users

Somehow its inconvenient just to have only certain folders auto-mounted.
If you want to mount specific folder while boot2docker is running you can do this with sshfs every time you restart boot2docker ( its not permanent!! )


Connecting to running container:

In Linux based environment like debian/ubunto you might not have hassle to connect to running containers, You can use

Or ssh directly to the container if you are running sshd inside.

But in MAC/Windows its little difficult because you need to ssh into the VM (boot2docker) and then ssh or docker attach to the container, since docker is terminal program so many people have their own development environment and terminal profiles in their machines so when you ssh to the VM (boot2docker) is not so convenient because i don’t have my bash profile,my terminal PS1, and my ZSH for example! so to overcome this there is an awesome tool https://github.com/jpetazzo/nsenter, you need to install it:

Script that does it all for you !!


Docker Ports:
Docker ports on linux is easy same port you expose is the one you get in docker host, but in MAC/Windows its running in VM (boot2docker) which if you need to expose the ports from docker to the MAC/Windows you need to forward the ports from the VM to the MAC/Windows, again its not convenient.. So script the forward all the VM(boot2docker) Ports to the host OS ( run it once ):



Docker ENV variables:
When you link containers together, docker share some information about the linked container port etc.. in the env variables of the running container, Now when you run the container and use “docker-enter CONTAINER_NAME” as previous in this article, you will get in the container with user but no env variables, so small bash script that save all the env variables into file somewhere then you can source it when you get it.

Then when you get in you do the below and you get the env variables


Docker Port Mapping:
When you expose a lot containers ports, it become somehow difficult to get every container and check ports mapping between docker and boot2docker host (VM), Small bash script that make this very easy to do:

Output looks like below :


Clean docker images and containers:
When you work with a lot of docker images and builds, you always fixing image errors and rebuilding again and again till you get it to work as intended, you will be left with alot of images and containers that is not used and its not needed it was just part of you builds at some point, now when you have the final images working you might need to clean the previous stuff, doing this with docker require you to know the image and container ids in order to remove them, small bash script that allow you do this easily, just stick it in your path anywhere and run it


Docker Daemon Foreground Process:
In order to keep docker container running you need to have foreground process running, so you need to remove all daemonization of any process that you need to run if you want to run it in docker containers for example ( nginx, php-fpm, redis…. ), or you can use any sorta of provisioning like provisioner.
Another solution to keep the process running in container when you run it with -d ( daemon ) option is to add and run bootstrap file on running the container, example Dockerfile :

and in bootstrap.sh file you start your service and keep the process running using sleep command.


Changing configuration with Dockerfile:
When you build an images most of the times you need your own configuration for every software you are running, so changing the configuration is a must to do,
Adding configuration files then copy it with docker COPY/ADD command is not bad idea but you will need to maintain this complete files for newer version of that software even minor versions or changing some params in service /etc/init.d/ files. Better work with this is to use the text manipulation tools that will alter and change your configuration files for you and you won’t need to keep up to date versions of the complete files! so SED, AWK also VIM if your friends!

Example Dockerfile:

If you couldn’t get it to work with SED and AWK then VIM is perfect alternative:


Few notes:

  • ADD is Similar to COPY, the difference is ADD Will unzip the source to the destination if its .tar.gz for example.
  • Only one CMD Entry for Dockerfile
  • Only one ENTRYPOINT for Dockerfile

Finally the nice thing when you use docker its completely isolated environment, you can have lots of experiments and builds with all kinda of software without touching your OS, then when you done you can remove them! also you can simulate literally any kinda of complex infrastructure, you can use it as proof of concept and prototypes for your software production plans, …. etc.. lots of ideas will be built on docker, and definitely its fun software to work with.

I have tried to keep this short as possible as i also wanted to share my experience using docker! i will write more articles about real usage examples for docker in the future.

Google docker and you will find lots of tutorials and start guides that will help you real quick to learn it.

AWS Certified Solutions Architect – Associate

Today i have passed AWS Certified Solutions Architect – Associate Level, I am glad that i have done it, I have AWS Certified Solutions Architect – Professional and AWS Certified DevOps Certification Scheduled! Looking forward to get them done! its still way to go AWS!
I will write next couple of days a detailed post about preparation, Q/A, Hands on Experience etc..

AWS Certified Solutions Architect - Associate

AWS Certified Solutions Architect – Associate

Index HTML content pages in Apache Nutch 2.x ( 2.2.1 )

Index-html Plugin for apache nutch 2.x

I have looked almost all over the posts tutorials out there, I haven’t find complete module that works or clear approach! so i have build plugin that really WORKS !

Index HTML content of the pages in Apaache Nutch 2.x ( 2.2.1 )

Plugin is hosted on Github repo !

——————————————————

Instruction:

Compile from Source

Download the plugin folder “index-html” and copy it to you Apache nutch 2 plugin directory ( ex: apache-nutch-2.2.1/src/plugin )
Add the ( index-html ) plugin to The plugin folder build.xml ( apache-nutch-2.2.1/src/plugin/build.xml ) in target ( deploy and clean ) so the file will look like

Run ( ant runtime ) in apache nutch 2 root folder to start the build
You should have index-html.jar in build folder
Enable the plugin by adding it to nutch-sites.xml ( or nutch-default.xml ) like beloe :

The plugin will add new Field “rawcontent” to the Nutch Doc, To index this field you need to add it to ( scheme.xml or schema-solr4.xml ) like

Run the crawler and you should see the new field rawcontent in index!
Use Pre-Compiled Library

In The repo there is Build folder contain compiled .jar library ready for use.
Copy the library to your runtime path local if you are running the plugin locally ( apache-nutch-2.2.1/runtime/local/plugins ) Then follow the above steps to configure nutch-sites.xml

Screen Shot

Rawcontent screen shot

Parent Package/Theme Magento

Since Magento v. 1.9 / 1.14 a new XML file has been added to the theme structure which is very helpful building new theme and defining the parent ( fallback ) Package/theme!

theme.xml located in the etc folder and sample contents ( building new theme based on rwd theme )

File tree looks like

So if you got new theme based on different existing theme even in different packages! you can use this XML to define the fallback package/theme without the need to define it in the backend ” System -> Configuration -> Design ”

 

 

MAGENTO CERTIFIED SOLUTION SPECIALIST EXAM

During the Magento Imagine 2014 conference, Magento rolled out the new Magento Certified Solution Specialist exam as a part of Magento ecosystem and it is finally complimenting the certification endeavours of Magento with a non-developer kind of certificate.
I was lucky enough to pass and thus have decided to share some experience that might help to successfully navigate through the process.
just-certifed
First, and possibly the only thing you will need for the preparation at all, is believing in yourself the Study Guide. As outlined in there, questions in the exam are coming from the three areas of knowledge: General eCommerce, Magento Architecture, Elements of a Magento eCommerce Site, Application of Knowledge to Business Goals.

General eCommerce questions seemed like the easiest ones for me and should not be a problem for you, if you have worked in this area for a while. What I’d recommend to spend the time on, is going through admin and making sure that in particular you understand:

  • How the Price Rules work
  • What kind of configurations are available for Shipping and Tax
  • How does the native Checkout works
  • What do you know about the native Magento’s recurring billing functionality
  • How does the layered navigation works
  • How PayPal integrates with Magento
  • What are the Magento Enterprise specific features and how they can be used.
  • One of the main points is to understand what functionality is available in Magento natively, so you know what functionality would require customisations. You also need to be pretty much familiar with Magento Enterprise.

Overall, the exam is not too difficult and I’d say that someone who is used to perform routine operations in Magento Admin and knows Magento in general would likely pass the exam. So take the exam and you’ll have this bright mark added to your profile:

cert-logo-solution

PHP Synchronous and Asynchronous SoapClient

Asynchronous PHP Soap Client

* PHP does not support multithreading and asynchronous programming.

The purpose of this project is to have an Asynchronous SoapClient that allow us to perform multiple requests to SoapServer.

Almost all the companies that provide services using SoapServer and alot of them provide asynchronous Soap Api but Usually it very poor and it lack the real-time response and its poor comparing to the synchronous Soap Api.

So This class asynchronous soap php will allow you to be able to use different asynchronous Soap Implementation! that work with Synchronous Soap Api !

See the Examples to see how to use it.

Features

  • Client can work in Asynchronous (multi) and Synchronous (single) mode.
  • Multiple calls using curl_multi_exec, Doesn’t wait for soap consecutive calls ! This client will save alot of time and resources doing multiple requests at same time !
  • Each Request has hash id which is unique to each request ( If you execute the same request 100 times more, it will have the same hash ) so no duplicate requests
  • Very Easy to debug every single point during the request! also ability to use CURL_VERBOSE to debug the connections to the Soap Host
  • Very easy exception handling in async mode

Find the code is on GitHub Repo

Magento Error "You cannot define a correlation name 'link_attribute_position_int' more than once"

After Migration from old Magento Version to newer Magento Version,
Some times When you try to duplicate product from Catalog -> Products -> Duplicate
You get this Error

You cannot define a correlation name 'link_attribute_position_int' more than once

Well this error indicate that you have Duplicate attribute position link in Database table
catalog_product_link_attribute
So when the duplicate method try to duplicate the product it looks into the folder and duplicate the attributes in that table, By Default the attributes in that table are ( Position, Qty )

So to solve this Error check duplicate rows for link_type_id = 3, Query is
SELECT * FROM `catalog_product_link_attribute` WHERE `link_type_id` = '3'
Then Delete the duplicate rows from the Table and make sure it looks like the screen below

Exclude Files/Folder from commit to specific branch git

Using git hooks !
Here is working example ” I have tested it and made sure it works fine ”
Usage :

git add . // add all the files for example ( including the files you want to exclude )
git commit -am ‘Test pre-commit’ // commit all the files “pre-commit” hook will run

Directory Structure example

Create file in your git root directory and call it { .ignorefolders } with files or folders you want to exclude ” example below ”

I have added “/” at the beginning of each line as i wanted it to be relevant to the root dir
” You can add complete folders, individual files ”
Create file { pre-commit } in folder { .git/hooks } with content below

Notes :

Code is well documented ( You can use exit or return ) in the conditional statements depends on the logic you want !

Use-case :

If you add “git-add .” and all the files you added is excluded in the config file “.ignorefolders”, Then you try to commit!
The files will be removed from the commit but you will be able to commit an empty “without any changes” commit.

Use Single or Multiple ( Slave or Master ) MySQL Databases with Magento | Magento Optimization

I have decided to write a series of magento optmization and performance tuning, and i will start with this subject as first article.

This articles will be different from the ones you see it on the internet !! it will be unusual solutions, different, solving real problems and achievable on all hosting and development platforms !

Assuming you have correct replication setup of your databases. ( slave_1,slave_2,slave_3 etc… )

The below solution will allow you to Alternate customers to USE ( Read ) data from different databases so you don’t have high load on one database.

– Step 1:

In File
app/etc/config.xml

Find “core_read” Closing tag

Add After the closing tag ( as many as databases you want to use ) it should look like below:

– Step 2:
And Add the new connection in your apt/etc/local.xml after ( “/default_setup>” closing tag )

– Step 3:
Copy the File From ” app/code/core/Mage/Core/Model/Resource.php ” == TO ==> ” app/code/local/Mage/Core/Model/Resource.php ”

1- Find protected $_mappedTableNames;

2- Add this method below :

3- Modify the method ” public function getConnection($name) ” to Look like below :

This will allow you to use as many as databases you specify in the XML and PHP CODE for core_read connection and the default_setup connection for all other connections in magento ( core_write, core_setup )

Infinite Scroller Ajax Products Magento

Description:

This extension is for when the user reaches the end of the current product list, the next page is loaded automatically by AJAX after the end of the list. Easy to install and configure, this module works 100% out of the box with Magento’s default theme, and also gives to you the posibility to configurate the custom selectors of your custom theme. Now you can get a more friendly UI by helping the user to save clicks and time.
Tested and working with Magento CE 1.5+… 1.7.0.2 , and Magento EE 1.10+.. 1.13.0.1

Features:

* Minimize category page load time significantly in Magento!
* Very easy to configure and to adjust for different themes.
* Works with sorting/filters.
* Super blazing fast, light weight as its only load products without any other blocks/models/etc..

Installation:

  • Download from githubt repo or Magento Connect.
  • Configure the options for your theme on System / Configuration /Catalog / Ajax Products seccion.
  • Scroll to infinity and beyond!

Configuration:

If you have a different theme other than the default, you will need to copy the default theme files to your custom theme folder and configure the plugin by going to System / Configuration / Catalog / Ajax Products.

Screen shots

Category page

Configuration Backend

Get all product attributes for a magento object using MySQL

The below MySQL Query allow you to get all the product attributes for specific product by its sku or id, change “YOUR_SKU” to the product you want to get.

Add/Edit/Remove Regions in Magento address region/state/city field

Manage regions in magento addresses

Magento provides country and region* as dropdown option in addresses This module provides flexibility of managing this regions for each country with its locale translations.

Very Easy to use ( it allow you to edit in place all the values ) hover and click on the value so you will be able to edit in place ( look at the screen shots )

Test and working with Magento CE 1.5+… 1.7.0.2 , and Magento EE 1.10+.. 1.13.0.1

  • Usage :1 – Install the Module2 – Go to Menu CMS -> Country States

You can download it from Magento Connect website
http://www.magentocommerce.com/magento-connect/catalog/product/view/id/18954/

OR
Check the module on github HERE https://github.com/Meabed/magento-regions-manager

Check the scren shots:

Inline Edit

Edit Region

Inline Edit

Grid List of regions

You will see the grid and how easy to modify the Regions/state for each country and its translation for each locale in your system.

Promotions don't apply to downloadable files prices in Magento

Initial conditions:

  • Magento 1.7 installed (haven’t tried with previous versions)
  • One (downloadable) product with multiple downloadable files, with prices added to the default product (let’s say product that costs 50$ + 2 downloadable files, one free, the other an extra 50$ )
  • A new promotion (Catalog price rule) that applies to all products (let’s say -20%)

More info about promotion:

Applies to all products, all groups, is active and applied, applies ‘by percentage of original price’, enable discount for subproducts -> Yes, stop further rule for processing -> No
Expected result:

Price for the product with the 50$ file: 80$ (80% from 100$)

Actual result:

Price for the product with the 50$ file: 90$ (80% from the initial 50$, and the full price for the downloadable file).

Conclusion:

The promotion doesn’t apply to the extra prices that downloadable files have.

Question(s):

  • Is this the desired behavior for downloadable files? Or is this a bug ?
  • Any tips on how to modify the code (eventually create a module) to make it work as expected ? (Just tips, ie. what to extend)

Answer:

Links / downloadable files its not products entities ( so it doesn’t have price_index table and it doesn’t treated as products )

There is 2 Ways to apply promotion in products

1. Catalog Price Rules

2. Shopping Cart Price Rules

As your question stated that you used Catalog Price Rules I have solved your question using Catalog Price Rules.

Create Module and rewrite the Model

Mage_Downloadable_Model_Product_Type

======

and the Code Below calculate the price of each Link on the fly ( even if you have more than one rule applied to the same product )

* This not a bug, this is intended functionality as the linked product designed different from the catalog products.

Magento Adding existing attribute to all attribute sets

Question :
I have an existing attribute for an embed code. I need to associate this attribute with 120+ existing attribute sets.

If I know the attribute set id, how can I go about adding the attribute to all attribute sets programmatically?
========================================
Solution :

I have found it interesting to write code for this issue so here is the solution that works :)

Run this code in php script including mage.php and let me know if it works well.

replace ( firstname ) with the attribute code that you want to mass add to all attribute sets

Add Additional Payment Information In Admin -> Sales -> Orders -> View

Hi,
I have ran through a case when i was integrating payment method and I wanted to add additional information in adminhtml view order in sales.
I have found 2 ways of doing this:-
1- Add info.phtml ( Mage_Payment_Block_Info ) and set the template file

2- Adding Additional Payment Information in the payment table which i preferred because i didn’t want to create template etc…
So I implemented 2 methods with events as follow
config.xml

And in the Observer.php
The 2 methods implemented as follow :

Terminal PS1 Colors !

Among all the terminal colors out there my favorite one is :
Example Colors :

However Here is some useful information in the Manual :

Color escape code general syntax:

Foreground colors

Background colors are 40, 41, 42, etc.

Text styles:

Reset to plain (always put at the end!):

Available variables:

Set default category product position magento

So in one of my projects i wanted to change the default Category product position in the admin grid when you assign new product to category
The data sent as array of ( product_id => sort position ) and its been generated in from Javascript function in

So you can modify this function to assign the position input different default value when you check this product to be assigned to the category.
The modified function will look like this

Change decimal attribute to int in magento !

Hi,
I was doing some work for client, and i have seen they created a decimal attribute as currency and they were already using it as int ! so the one who created it didn’t know actually what he is doing. anyway i have decided to convert to INT and ofcourse keeping the same old values as its.

So here is MySQL script the convert the attribute ( in my case the attribute_id = 168 change it to be the attribute you want to change !)

CHANGE THE ATTRIBUTE_ID TO MATCH THE ATTRIBUTE ID YOU WANT TO CHANGE !!

——————

Then get the old values from decimal EAV table and insert them into the EAV int table

Run this code if you want to have DEFAULT value for all other PRODUCTS that have no value assigned to this attribute
In my case the default value = 10000 // CHANGE IT TO WHAT YOU NEED

Add a custom option to a quote item magento

Yes this is possible, you need to use the observer

I have been adding Delivery date with each product in the orders

So you can change this to be the Option you want to add to each product or so.

———-