Take this with a bit of salt, as I can not seem to find it anymore, but....
I recall having read something in relation to the GDPR passing, that not even an individual is allowed to hold personal information on someone else, even if it is just through "contacts" on your phone (With permission, you can of course).
Therr might be confusion between being allowed to remember information and passing on information. The private address book can contain entries. But giving apps access to that supposedly private data may be passing on data of others without their consent.
Granting whatsapp (or any other app that sends home a copy of the address book) access to the address book without the consent of everyone stored there might be a violation.
> (18) This Regulation does not apply to the processing of personal data by a natural person in the course of a purely personal or household activity and thus with no connection to a professional or commercial activity. Personal or household activities could include correspondence and the holding of addresses, or social networking and online activity undertaken within the context of such activities. However, this Regulation applies to controllers or processors which provide the means for processing personal data for such personal or household activities.
Reminder that, "articles" are the regulations themselves, and "recitals" are kind of supplementary FAQ-style clarifications about how/when to apply the articles.