Statistical data analysis should always be done with care if outliers are present in the data, since they have the potential to spoil the analysis. However, usually it is not clear if multivariate data contain outliers, and in particular, if such outliers would affect the statistical method to be used. Diagnostic plots of the results from the analysis will only reveal outliers if the method itself is robust against the outliers. Moreover, the impact of outliers depends on the statistical model being used. Identifying outliers in compositional data is even more tricky because their values are unusual not in the absolute but in a relative sense. With the log-ratio approach for compositional data analysis, outliers could even be artificially created by including variables with extremely low and unreliable values - a frequent practical issue. We will discuss these problems and provide more detailed insight, propose some possible approaches to cope with these issues, and illustrate them at real data, mainly from the field of geochemistry.