Estimating tree fruit production is important for various purposes; however, no consensus has been established on which method is appropriate over different tree species. Here the aim of this study was to assess whether the tree crop size of taxa with different fruit morphology could be accurately estimated using the visual survey method (VSM), taking into consideration errors due to the participation of different observers. Eight main tree species of temperate deciduous forests of Japan were used. Using a hierarchical Bayesian framework, the model parameters were estimated to convert the number of fruits counted by VSM to the fallen fruit density. The estimated model had a high accuracy for seven of the eight tree species. The trees of the outlier species had the smallest diameter and height, presumably resulting in false-positive errors. The model could be easily generalized for other species that were not tested in this study by adjusting the model parameters to the size of the unit used in the visual survey. The bias caused by errors depending on the observer (likely false-negatives) was not substantial. Thus, the generalized VSM proposed in this study should be useful to accurately estimate the crop size across various tree taxa with low workload.