You have probably noticed patches of what looks like fungus or mold growing on the the trunks and branches of your trees. These are actually lichens; symbiotic colonies of fungi, algae and/or cyanobacteria.
The fungi provides housing, giving the algae/cyanobacteria areas in which to grow that they could not normally inhabit. In trade, the algae/cyanobacteria use photosynthesis to provide carbohydrates (food) to the fungi.
In general, they are nothing to worry about.