Brown spot of passion fruit is caused by the fungi Alternaria passiflorae and Alternaria tenuis and can be very damaging, particularly to purple passion fruit which is more susceptible to the disease than yellow passion fruit varieties. The fungus causes circular brown spots on leaves which enlarge and can cause leaves to drop from the tree prematurely. When the fungus infects the fruits it can cause severe economic losses. The disease appears as tiny specks on the fruit which enlarge into sunken circular lesions with a brown center. The fruit shrivels and drops from the vine.
Brown spot is common on purple passion fruit in New Zealand and East Africa but is more common on yellow passion fruit in the US state of Hawaii. Which variety are you growing and where?
Control of brown spot relies on cultural practices such as allowing adequate space between plants, pruning back leaders to open the canopy, thus reducing humidity and promoting good air circulation around the foliage, and good sanitation practices e.g. destroying crop debris and disinfecting tools and equipment regularly.
In terms of fungicide application, protective copper sprays can be used for organic production. Certain chemicals, such as mancozeb, have been shown to be effective against the fungus but are not approved by the FDA for use in the US. That is why it is important to tell us your geographic location...