The aim of this work is to find the best local binary pattern for describing a given texture in order to meet the needs of effectiveness in the domain of fabric color card image retrieval. Firstly, four different local binary pattern (LBP) descriptors are introduced. We are conducted in-depth research on four local binary patterns, including conventional LBP, rotation invariant LBP, uniform pattern LBP and rotation invariant uniform patterns LBP. Secondly, texture feature extraction algorithms based on the four LBP descriptors are designed and a corresponding approach to retrieving the fabric color card images is proposed by using local binary pattern. The proposed retrieval algorithm is used for finding k-nearest neighborhoods that match each tested image. Finally, the proposed retrieval approaches based on four different LBP modes are tested and compared on the public image texture data sets and the real fabric color card data set in details. By analyzing the experimental results, we select an approach that performs well on the real fabric color card data set, which can satisfy the actual need of fabric color card retrieval.