Zappone said the criticism of the appointment in recent days has impacted the legitimacy of the job itself.
There had been controversy about the Government appointing a former Minister and Zappone's recent hosting of an event for 50 people at the Merrion Hotel in Dublin.
Katherine Zappone said the role of Special Envoy on Freedom of Opinion and Expression could only be acceptable and valuable to the country if the appointment had the support of all parties.