Changes in editable template policies not reflected in existing pages

I added a new component to my editable template by source code, editing 

ui.content\src\main\content\jcr_root\conf\my-site\settings\wcm\templates\my-template\structure\.content.xml

and

ui.content\src\main\content\jcr_root\conf\my-site\settings\wcm\templates\my-template\policies\.content.xml

I added an embed component.

And it's being added to all the pages.

However, in new created pages the component shows a list of embeddables whereas in already existing pages it does not.

In new pages:

In already existing pages:

I know embeddable options are controlled by policies, which are working because the radio button is restricted to "Embeddable / None" via policy (there are more values). I checked both pages using this template.

What could be the cause for this?

We use AEM as a Cloud Service.

Thanks.