The major use of brick masonry units in the U.S. is in facades for office buildings and single family homes. The traditional method for building masonry walls is on-site by bricklayers. An alternative method is panelization or prefabrication of brick panels in a plant environment. While many mechanical problems exist, the real-time control represents a challenging aspect of robotizing brick masonry work. This paper presents an effort to study the development of an experimental robotic masonry system and its relevant control modules. It also describes two control frameworks for different levels in a robotic brick masonry prototype: (a) local control, and (b) global control. The local control system includes three work cells: (a) gripping and handling, (b) quality control, and (c) brick placement. First, the components and equipment used in a work cell are listed, then the experimental work performed with them is discussed to show how sensors are used for dehacking, brick placement, brick color detection, and brick size measurements. The paper then continues to describe a global control system that will integrate the three local work cells utilizing a hierarchical structure.