TarMK replication is not working | Community
Skip to main content
S
Swathi1
New Participant
August 17, 2016
Solved

TarMK replication is not working

  • August 17, 2016
  • 11 replies
  • 4132 views

Hi All,

We use tarMk replication process to replicate from our Primary author to two stand-by authors.We had following activities going on the servers.

  • Back of the dam assets. They were built in small packages and were deleted after downloading.
  • Ran offline Compaction successfully on the three servers.
  • Auto injection of the pages.

At the time of auto-injection, we had noticed that tarmk-log is stopped. The replication was stopped for more than 10 hours completely. So, we recycled all the three servers. Replication went well for couple of hours after restarting but again stopped. Below are the tarMk logs 

16.08.2016 20:17:52.759 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK closed: <path>/repos/segmentstore

16.08.2016 20:17:52.759 *ERROR* [defaultEventExecutorGroup-5-1] org.apache.jackrabbit.oak.plugins.segment.standby.client.StandbyClientHandler Exception caught, closing channel.

java.lang.IllegalStateException: null

        at com.google.common.base.Preconditions.checkState(Preconditions.java:134)

        at org.apache.jackrabbit.oak.plugins.segment.file.TarWriter.containsEntry(TarWriter.java:167)

        at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.containsSegment(FileStore.java:814)

        at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.containsSegment(FileStore.java:803)

        at org.apache.jackrabbit.oak.plugins.segment.standby.store.StandbyStore.readSegment(StandbyStore.java:93)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:136)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:108)

        at org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:79)

        at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:447)

        at org.apache.jackrabbit.oak.plugins.segment.standby.client.SegmentLoaderHandler.initSync(SegmentLoaderHandler.java:105)

        at org.apache.jackrabbit.oak.plugins.segment.standby.client.SegmentLoaderHandler.channelActive(SegmentLoaderHandler.java:78)

        at org.apache.jackrabbit.oak.plugins.segment.standby.client.StandbyClientHandler.setHead(StandbyClientHandler.java:105)

        at org.apache.jackrabbit.oak.plugins.segment.standby.client.StandbyClientHandler.channelRead0(StandbyClientHandler.java:77)

        at org.apache.jackrabbit.oak.plugins.segment.standby.client.StandbyClientHandler.channelRead0(StandbyClientHandler.java:39)

        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)

        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)

        at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)

        at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)

        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)

        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)

        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)

        at java.lang.Thread.run(Thread.java:745)

16.08.2016 20:37:58.132 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component still not activated. Ignoring the initialization call

16.08.2016 20:37:58.160 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Initializing SegmentNodeStore with BlobStore [DataStore backed BlobStore [org.apache.jackrabbit.oak.plugins.blob.datastore.OakFileDataStore]]

16.08.2016 20:37:58.322 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened: <path>/repos/segmentstore (mmap=true)

16.08.2016 20:37:58.473 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService SegmentNodeStore initialized

16.08.2016 20:37:59.134 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.segment.standby.store.StandbyStoreService started standby sync with 159.231.172.44:40901 at 5 sec.

16.08.2016 22:53:21.200 *ERROR* [defaultEventExecutorGroup-4-1] org.apache.jackrabbit.oak.plugins.segment.standby.client.SegmentLoaderHandler Exception caught, closing channel.

io.netty.handler.timeout.ReadTimeoutException: null

Please advice

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by AnkurAhlawat-1

Hi Swathi,

I have tried to replicate mentioned scenario below are my findings:-

Test Case:- Ran offline tar compaction on standby

Result:- After 3 days sync not started, standby size is lower than primary. 

So, as advised by opkar it is better and safe to copy paste primary repo to standby instance.

Note:- I am using AEM 6.1

11 replies

Show previous replies
O
ogill
Employee
August 17, 2016

Hi Swathi,

first let's sort out the terminology, it's not replication that is used in Cold Standby setup. There is a sync process that is initiated from the standby to pull the latest head revision from the primary instance. 

If you perform any heavy activity on your primary server, I would recommend you wait for this to complete before stopping the primary instance to perform offline compaction.

Unfortunately the docs are incorrect and you should not be running offline compaction on the standby instances. Instead, as when you are installing a hot fix or service pack, you should re-create your standby instances after running offline compaction on the primary instance. This is error had been raised internally and the docs team should be updating the documentation shortly.

There have been cases in the past where it takes a long time for the standby to synch with the primary after running offline compaction on the primary, which is why it's recommended to re-create the standby. Another issues was that the standby instance would grow several factors larger than the primary instance.

Regards,

Opkar

[0]https://docs.adobe.com/docs/en/aem/6-2/deploy/recommended-deploys/tarmk-cold-standby.html

Terms & ConditionsAccessibility statement

Loading footer...