Taking a second opinion is a normal and sensible part of medical care, especially after unsuccessful cycles. A good doctor should not be offended, and you do not need anyone's permission to consult elsewhere.
Switching rarely means starting from zero. Most test results remain useful for a while, though some (like infection screening or hormone levels) may need repeating after a few months. What helps most is a complete record of what was done: your stimulation details, how many eggs were collected, fertilisation and embryo development notes, transfer details and any procedure reports. You have a right to copies of your own medical records, so ask for them politely in writing.
If you have frozen embryos, ask your current clinic how transfer to another centre works; it involves consent forms and careful transport, so plan it before deciding.
When you meet a new doctor, ask what they would do differently and why. Sometimes the answer is a genuine change in approach, and sometimes it's reassurance that the plan was reasonable. Either is valuable. Do talk to a doctor before stopping any medicines you're currently on.