Index HTML content pages in Apache Nutch 2.x ( 2.2.1 )

Index-html Plugin for apache nutch 2.x

I have looked almost all over the posts tutorials out there, I haven’t find complete module that works or clear approach! so i have build plugin that really WORKS !

Index HTML content of the pages in Apaache Nutch 2.x ( 2.2.1 )

Plugin is hosted on Github repo !



Compile from Source

Download the plugin folder “index-html” and copy it to you Apache nutch 2 plugin directory ( ex: apache-nutch-2.2.1/src/plugin )
Add the ( index-html ) plugin to The plugin folder build.xml ( apache-nutch-2.2.1/src/plugin/build.xml ) in target ( deploy and clean ) so the file will look like

Run ( ant runtime ) in apache nutch 2 root folder to start the build
You should have index-html.jar in build folder
Enable the plugin by adding it to nutch-sites.xml ( or nutch-default.xml ) like beloe :

The plugin will add new Field “rawcontent” to the Nutch Doc, To index this field you need to add it to ( scheme.xml or schema-solr4.xml ) like

Run the crawler and you should see the new field rawcontent in index!
Use Pre-Compiled Library

In The repo there is Build folder contain compiled .jar library ready for use.
Copy the library to your runtime path local if you are running the plugin locally ( apache-nutch-2.2.1/runtime/local/plugins ) Then follow the above steps to configure nutch-sites.xml

Screen Shot

Rawcontent screen shot

Parent Package/Theme Magento

Since Magento v. 1.9 / 1.14 a new XML file has been added to the theme structure which is very helpful building new theme and defining the parent ( fallback ) Package/theme!

theme.xml located in the etc folder and sample contents ( building new theme based on rwd theme )

File tree looks like

So if you got new theme based on different existing theme even in different packages! you can use this XML to define the fallback package/theme without the need to define it in the backend ” System -> Configuration -> Design ”




During the Magento Imagine 2014 conference, Magento rolled out the new Magento Certified Solution Specialist exam as a part of Magento ecosystem and it is finally complimenting the certification endeavours of Magento with a non-developer kind of certificate.
I was lucky enough to pass and thus have decided to share some experience that might help to successfully navigate through the process.
First, and possibly the only thing you will need for the preparation at all, is believing in yourself the Study Guide. As outlined in there, questions in the exam are coming from the three areas of knowledge: General eCommerce, Magento Architecture, Elements of a Magento eCommerce Site, Application of Knowledge to Business Goals.

General eCommerce questions seemed like the easiest ones for me and should not be a problem for you, if you have worked in this area for a while. What I’d recommend to spend the time on, is going through admin and making sure that in particular you understand:

  • How the Price Rules work
  • What kind of configurations are available for Shipping and Tax
  • How does the native Checkout works
  • What do you know about the native Magento’s recurring billing functionality
  • How does the layered navigation works
  • How PayPal integrates with Magento
  • What are the Magento Enterprise specific features and how they can be used.
  • One of the main points is to understand what functionality is available in Magento natively, so you know what functionality would require customisations. You also need to be pretty much familiar with Magento Enterprise.

Overall, the exam is not too difficult and I’d say that someone who is used to perform routine operations in Magento Admin and knows Magento in general would likely pass the exam. So take the exam and you’ll have this bright mark added to your profile:


PHP Synchronous and Asynchronous SoapClient

Asynchronous PHP Soap Client

* PHP does not support multithreading and asynchronous programming.

The purpose of this project is to have an Asynchronous SoapClient that allow us to perform multiple requests to SoapServer.

Almost all the companies that provide services using SoapServer and alot of them provide asynchronous Soap Api but Usually it very poor and it lack the real-time response and its poor comparing to the synchronous Soap Api.

So This class asynchronous soap php will allow you to be able to use different asynchronous Soap Implementation! that work with Synchronous Soap Api !

See the Examples to see how to use it.


  • Client can work in Asynchronous (multi) and Synchronous (single) mode.
  • Multiple calls using curl_multi_exec, Doesn’t wait for soap consecutive calls ! This client will save alot of time and resources doing multiple requests at same time !
  • Each Request has hash id which is unique to each request ( If you execute the same request 100 times more, it will have the same hash ) so no duplicate requests
  • Very Easy to debug every single point during the request! also ability to use CURL_VERBOSE to debug the connections to the Soap Host
  • Very easy exception handling in async mode

Find the code is on GitHub Repo

Magento Error "You cannot define a correlation name 'link_attribute_position_int' more than once"

After Migration from old Magento Version to newer Magento Version,
Some times When you try to duplicate product from Catalog -> Products -> Duplicate
You get this Error

You cannot define a correlation name 'link_attribute_position_int' more than once

Well this error indicate that you have Duplicate attribute position link in Database table
So when the duplicate method try to duplicate the product it looks into the folder and duplicate the attributes in that table, By Default the attributes in that table are ( Position, Qty )

So to solve this Error check duplicate rows for link_type_id = 3, Query is
SELECT * FROM catalog_product_link_attribute WHERE link_type_id = '3'
Then Delete the duplicate rows from the Table and make sure it looks like the screen below

Exclude Files/Folder from commit to specific branch git

Using git hooks !
Here is working example ” I have tested it and made sure it works fine ”
Usage :

git add . // add all the files for example ( including the files you want to exclude )
git commit -am ‘Test pre-commit’ // commit all the files “pre-commit” hook will run

Directory Structure example

Create file in your git root directory and call it { .ignorefolders } with files or folders you want to exclude ” example below ”

I have added “/” at the beginning of each line as i wanted it to be relevant to the root dir
” You can add complete folders, individual files ”
Create file { pre-commit } in folder { .git/hooks } with content below

Notes :

Code is well documented ( You can use exit or return ) in the conditional statements depends on the logic you want !

Use-case :

If you add “git-add .” and all the files you added is excluded in the config file “.ignorefolders”, Then you try to commit!
The files will be removed from the commit but you will be able to commit an empty “without any changes” commit.

Use Single or Multiple ( Slave or Master ) MySQL Databases with Magento | Magento Optimization

I have decided to write a series of magento optmization and performance tuning, and i will start with this subject as first article.

This articles will be different from the ones you see it on the internet !! it will be unusual solutions, different, solving real problems and achievable on all hosting and development platforms !

Assuming you have correct replication setup of your databases. ( slave_1,slave_2,slave_3 etc… )

The below solution will allow you to Alternate customers to USE ( Read ) data from different databases so you don’t have high load on one database.

– Step 1:

In File

Find “core_read” Closing tag

Add After the closing tag ( as many as databases you want to use ) it should look like below:

– Step 2:
And Add the new connection in your apt/etc/local.xml after ( “/default_setup>” closing tag )

– Step 3:
Copy the File From ” app/code/core/Mage/Core/Model/Resource.php ” == TO ==> ” app/code/local/Mage/Core/Model/Resource.php ”

1- Find protected $_mappedTableNames;

2- Add this method below :

3- Modify the method ” public function getConnection($name) ” to Look like below :

This will allow you to use as many as databases you specify in the XML and PHP CODE for core_read connection and the default_setup connection for all other connections in magento ( core_write, core_setup )

Infinite Scroller Ajax Products Magento


This extension is for when the user reaches the end of the current product list, the next page is loaded automatically by AJAX after the end of the list. Easy to install and configure, this module works 100% out of the box with Magento’s default theme, and also gives to you the posibility to configurate the custom selectors of your custom theme. Now you can get a more friendly UI by helping the user to save clicks and time.
Tested and working with Magento CE 1.5+… , and Magento EE 1.10+..


* Minimize category page load time significantly in Magento!
* Very easy to configure and to adjust for different themes.
* Works with sorting/filters.
* Super blazing fast, light weight as its only load products without any other blocks/models/etc..


  • Download from githubt repo or Magento Connect.
  • Configure the options for your theme on System / Configuration /Catalog / Ajax Products seccion.
  • Scroll to infinity and beyond!


If you have a different theme other than the default, you will need to copy the default theme files to your custom theme folder and configure the plugin by going to System / Configuration / Catalog / Ajax Products.

Screen shots

Category page

Configuration Backend

Get all product attributes for a magento object using MySQL

The below MySQL Query allow you to get all the product attributes for specific product by its sku or id, change “YOUR_SKU” to the product you want to get.

Add/Edit/Remove Regions in Magento address region/state/city field

Manage regions in magento addresses

Magento provides country and region* as dropdown option in addresses This module provides flexibility of managing this regions for each country with its locale translations.

Very Easy to use ( it allow you to edit in place all the values ) hover and click on the value so you will be able to edit in place ( look at the screen shots )

Test and working with Magento CE 1.5+… , and Magento EE 1.10+..

  • Usage :1 – Install the Module2 – Go to Menu CMS -> Country States

You can download it from Magento Connect website

Check the module on github HERE

Check the scren shots:

Inline Edit

Edit Region

Inline Edit

Grid List of regions

You will see the grid and how easy to modify the Regions/state for each country and its translation for each locale in your system.

Promotions don't apply to downloadable files prices in Magento

Initial conditions:

  • Magento 1.7 installed (haven’t tried with previous versions)
  • One (downloadable) product with multiple downloadable files, with prices added to the default product (let’s say product that costs 50$ + 2 downloadable files, one free, the other an extra 50$ )
  • A new promotion (Catalog price rule) that applies to all products (let’s say -20%)

More info about promotion:

Applies to all products, all groups, is active and applied, applies ‘by percentage of original price’, enable discount for subproducts -> Yes, stop further rule for processing -> No
Expected result:

Price for the product with the 50$ file: 80$ (80% from 100$)

Actual result:

Price for the product with the 50$ file: 90$ (80% from the initial 50$, and the full price for the downloadable file).


The promotion doesn’t apply to the extra prices that downloadable files have.


  • Is this the desired behavior for downloadable files? Or is this a bug ?
  • Any tips on how to modify the code (eventually create a module) to make it work as expected ? (Just tips, ie. what to extend)


Links / downloadable files its not products entities ( so it doesn’t have price_index table and it doesn’t treated as products )

There is 2 Ways to apply promotion in products

1. Catalog Price Rules

2. Shopping Cart Price Rules

As your question stated that you used Catalog Price Rules I have solved your question using Catalog Price Rules.

Create Module and rewrite the Model



and the Code Below calculate the price of each Link on the fly ( even if you have more than one rule applied to the same product )

* This not a bug, this is intended functionality as the linked product designed different from the catalog products.

Magento Adding existing attribute to all attribute sets

Question :
I have an existing attribute for an embed code. I need to associate this attribute with 120+ existing attribute sets.

If I know the attribute set id, how can I go about adding the attribute to all attribute sets programmatically?
Solution :

I have found it interesting to write code for this issue so here is the solution that works :)

Run this code in php script including mage.php and let me know if it works well.

replace ( firstname ) with the attribute code that you want to mass add to all attribute sets

Add Additional Payment Information In Admin -> Sales -> Orders -> View

I have ran through a case when i was integrating payment method and I wanted to add additional information in adminhtml view order in sales.
I have found 2 ways of doing this:-
1- Add info.phtml ( Mage_Payment_Block_Info ) and set the template file

2- Adding Additional Payment Information in the payment table which i preferred because i didn’t want to create template etc…
So I implemented 2 methods with events as follow

And in the Observer.php
The 2 methods implemented as follow :

Terminal PS1 Colors !

Among all the terminal colors out there my favorite one is :
Example Colors :

However Here is some useful information in the Manual :

Color escape code general syntax:

Foreground colors

Background colors are 40, 41, 42, etc.

Text styles:

Reset to plain (always put at the end!):

Available variables:

Set default category product position magento

So in one of my projects i wanted to change the default Category product position in the admin grid when you assign new product to category
The data sent as array of ( product_id => sort position ) and its been generated in from Javascript function in

So you can modify this function to assign the position input different default value when you check this product to be assigned to the category.
The modified function will look like this

Change decimal attribute to int in magento !

I was doing some work for client, and i have seen they created a decimal attribute as currency and they were already using it as int ! so the one who created it didn’t know actually what he is doing. anyway i have decided to convert to INT and ofcourse keeping the same old values as its.

So here is MySQL script the convert the attribute ( in my case the attribute_id = 168 change it to be the attribute you want to change !)



Then get the old values from decimal EAV table and insert them into the EAV int table

Run this code if you want to have DEFAULT value for all other PRODUCTS that have no value assigned to this attribute
In my case the default value = 10000 // CHANGE IT TO WHAT YOU NEED

Add a custom option to a quote item magento

Yes this is possible, you need to use the observer

I have been adding Delivery date with each product in the orders

So you can change this to be the Option you want to add to each product or so.


Get EAV attribute id,name,.. from attribute code in magento

Something useful we always use if you want to join some attribute or sort by etc.. you need to get the attribute info! and this is helper function to do this.

MIGS Payment Integration with Magento

Three months back i have written magento payment module integration with MIGS because this merchant is most used in Middle East ( arab country ),

It has option to auto-check payments and update orders accordingly through migs API.

You can find it in Github.

I am also working on Cashu CPSP Integration with magento, I will release it soon !

contribution and comments are most welcome.