Getting nothing back, UK minister says frustration is growing with EU