Building

Started building at 2021/12/25 01:32:05
Calculating base
Updating mirror
Basing run on 7.1.0-1981 6aafc105b
Updating tree for run 25.12.2021-01.32
gocb is at 635fa6c, changes since last good build: none
libcouchbase is at f6822fc, changes since last good build: none
indexing is at 8d3522b, changes since last good build: none
kv_engine is at cda4d73, changes since last good build: none
ns_server is at fadbef1, changes since last good build: none
couchstore is at 0bf3a4b, changes since last good build: none
couchdb is at fca2e6b, changes since last good build: none
testrunner is at a23e488, changes since last good build: none
Switching eventing to unstable
eventing is at ee5930d, changes since last good build: none
Switching eventing-ee to unstable
eventing-ee is at 80089ff, changes since last good build: none
Pulling in uncommitted change 160397 at refs/changes/97/160397/3
Total 23 (delta 15), reused 17 (delta 15)
[unstable 3caccee0] MB-100 : Do not fail n1ql timeout test if we are catching timeout exceptions successfully
 Author: abhijpes 
 Date: Mon Aug 30 08:24:38 2021 +0530
 1 file changed, 2 insertions(+), 2 deletions(-)
Pulling in uncommitted change 163449 at refs/changes/49/163449/1
Total 48 (delta 27), reused 39 (delta 27)
[unstable 31e229c8] MB-100: collect data when test fails
 Author: AnkitPrabhu 
 Date: Tue Oct 12 14:02:31 2021 +0530
 1 file changed, 2 insertions(+)
Pulling in uncommitted change 166698 at refs/changes/98/166698/3
Total 4 (delta 3), reused 4 (delta 3)
[unstable 315006cd] MB-48334 : Fail rebalance unless all functions deployed on current encryption level
 Author: abhijpes 
 Date: Wed Dec 1 14:51:29 2021 +0530
 1 file changed, 9 insertions(+)
Building cmakefiles and deps
Building main product

Testing

Started testing at 2021/12/25 01:46:39
Testing mode: rebalance,n1ql,functional
Starting server
Starting server: attempt 1

Functional tests

2021/12/25 01:48:07 Started functional
Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:37556->127.0.0.1:9000: read: connection reset by peer
Quota setup Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:37556->127.0.0.1:9000: read: connection reset by peer
2021/12/25 01:48:16 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/25 01:48:21 Error in adding nodes : , response: {"otpNode":"n_1@127.0.0.1"}
2021/12/25 01:48:21 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 01:48:27 Removing node(s): [] from the cluster
2021/12/25 01:48:32 Rebalance progress: 0
2021/12/25 01:48:37 Rebalance progress: 0.1808449074074074
2021/12/25 01:48:42 Rebalance progress: 4.720052083333333
2021/12/25 01:48:47 Rebalance progress: 9.25925925925926
2021/12/25 01:48:52 Rebalance progress: 12.96657986111111
2021/12/25 01:48:57 Rebalance progress: 17.74088541666667
2021/12/25 01:49:02 Rebalance progress: 21.01417824074074
2021/12/25 01:49:07 Rebalance progress: 25.64380787037037
2021/12/25 01:49:12 Rebalance progress: 100
2021/12/25 01:49:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:49:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:49:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:50:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:50:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:50:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:50:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:51:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:51:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:51:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:51:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:52:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:52:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:52:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:52:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:53:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:53:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:53:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/25 01:53:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
=== RUN   TestAdvancedGetOps
2021/12/25 01:54:02 Deploying app: TestAdvancedGetOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 01:54:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps stored in temp store"
}
2021/12/25 01:54:09 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/25 01:54:14 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/25 01:54:19 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/25 01:54:24 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/25 01:54:24 App: TestAdvancedGetOps got deployed
2021/12/25 01:54:24 Starting bucket ops to source bucket
2021/12/25 01:54:29 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 01:54:34 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 01:54:34 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 01:54:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 01:54:34 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 01:54:39 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/25 01:54:44 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/25 01:54:49 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/25 01:54:54 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/25 01:54:54 App: TestAdvancedGetOps got un-deployed
2021/12/25 01:54:54 No eventing-con process running
2021/12/25 01:54:54 Function: TestAdvancedGetOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps deleting in background"
}
2021/12/25 01:54:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:55:06 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 01:55:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:55:07 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:55:17 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 01:55:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedGetOps (75.14s)
=== RUN   TestAdvancedInsertOps
2021/12/25 01:55:17 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 01:55:19 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/25 01:55:24 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:55:29 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:55:34 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:55:39 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:55:39 App: TestAdvancedInsertOps got deployed
2021/12/25 01:55:39 Starting bucket ops to source bucket
2021/12/25 01:55:45 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/25 01:55:50 Waiting for dst bucket item count to get to: 1024 curr count: 373
2021/12/25 01:55:55 Waiting for dst bucket item count to get to: 1024 curr count: 885
2021/12/25 01:55:55 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/25 01:55:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 01:55:55 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 01:56:00 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:56:05 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:56:10 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:56:15 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:56:15 App: TestAdvancedInsertOps got un-deployed
2021/12/25 01:56:15 No eventing-con process running
2021/12/25 01:56:15 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/25 01:56:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:56:26 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 01:56:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:56:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:56:38 Waiting for dst bucket item count to get to: 0 curr count: 766
2021/12/25 01:56:43 Waiting for dst bucket item count to get to: 0 curr count: 254
2021/12/25 01:56:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:56:43 Testing insert operation on source bucket
2021/12/25 01:56:43 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 01:56:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/25 01:56:49 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:56:54 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:56:59 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:57:04 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/25 01:57:04 App: TestAdvancedInsertOps got deployed
2021/12/25 01:57:04 Starting bucket ops to source bucket
2021/12/25 01:57:10 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/25 01:57:15 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/25 01:57:15 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/25 01:57:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 01:57:15 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 01:57:20 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:57:25 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:57:30 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:57:35 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/25 01:57:35 App: TestAdvancedInsertOps got un-deployed
2021/12/25 01:57:35 No eventing-con process running
2021/12/25 01:57:36 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/25 01:57:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:57:47 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/25 01:57:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:57:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:57:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertOps (156.50s)
=== RUN   TestAdvancedUpsertOps
2021/12/25 01:57:54 Starting bucket ops to source bucket
2021/12/25 01:57:55 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 01:57:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/25 01:58:01 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:58:06 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:58:11 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:58:16 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:58:16 App: TestAdvancedUpsertOps got deployed
2021/12/25 01:58:16 Starting bucket ops to source bucket
2021/12/25 01:58:22 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 01:58:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:58:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 01:58:22 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 01:58:27 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:58:32 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:58:37 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:58:42 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:58:42 App: TestAdvancedUpsertOps got un-deployed
2021/12/25 01:58:42 No eventing-con process running
2021/12/25 01:58:42 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/25 01:58:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:58:54 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 01:58:59 Waiting for dst bucket item count to get to: 0 curr count: 512
2021/12/25 01:58:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:59:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:59:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:59:05 Testing upsert operation on source bucket
2021/12/25 01:59:05 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 01:59:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/25 01:59:12 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:59:17 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:59:22 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:59:27 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/25 01:59:27 App: TestAdvancedUpsertOps got deployed
2021/12/25 01:59:27 Starting bucket ops to source bucket
2021/12/25 01:59:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:59:28 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 01:59:28 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 01:59:33 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:59:38 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:59:43 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:59:48 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/25 01:59:48 App: TestAdvancedUpsertOps got un-deployed
2021/12/25 01:59:48 No eventing-con process running
2021/12/25 01:59:48 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/25 01:59:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 01:59:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 01:59:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:00:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedUpsertOps (127.84s)
=== RUN   TestAdvancedReplaceOps
2021/12/25 02:00:02 Starting bucket ops to source bucket
2021/12/25 02:00:03 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:00:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/25 02:00:09 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:00:14 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:00:19 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:00:24 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:00:24 App: TestAdvancedReplaceOps got deployed
2021/12/25 02:00:24 Starting bucket ops to source bucket
2021/12/25 02:00:30 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 02:00:35 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 02:00:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:00:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:00:35 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:00:40 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:00:45 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:00:50 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:00:55 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:00:55 App: TestAdvancedReplaceOps got un-deployed
2021/12/25 02:00:55 No eventing-con process running
2021/12/25 02:00:55 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/25 02:00:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:01:07 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 02:01:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:01:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:01:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:01:14 Testing upsert operation on source bucket
2021/12/25 02:01:14 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:01:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/25 02:01:20 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:01:25 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:01:30 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:01:35 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/25 02:01:35 App: TestAdvancedReplaceOps got deployed
2021/12/25 02:01:35 Starting bucket ops to source bucket
2021/12/25 02:01:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:01:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:01:36 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:01:41 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:01:46 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:01:51 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:01:56 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/25 02:01:56 App: TestAdvancedReplaceOps got un-deployed
2021/12/25 02:01:56 No eventing-con process running
2021/12/25 02:01:56 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/25 02:01:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:02:08 Waiting for dst bucket item count to get to: 0 curr count: 203
2021/12/25 02:02:13 Waiting for dst bucket item count to get to: 0 curr count: 40
2021/12/25 02:02:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:02:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:02:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedReplaceOps (137.88s)
=== RUN   TestAdvancedDeleteOps
2021/12/25 02:02:20 Starting bucket ops to source bucket
2021/12/25 02:02:20 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:02:21 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/25 02:02:26 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:02:31 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:02:36 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:02:41 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:02:41 App: TestAdvancedDeleteOps got deployed
2021/12/25 02:02:41 Starting bucket ops to source bucket
2021/12/25 02:02:46 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:02:51 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:02:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:02:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:02:51 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:02:56 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:03:01 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:03:06 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:03:11 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:03:11 App: TestAdvancedDeleteOps got un-deployed
2021/12/25 02:03:11 No eventing-con process running
2021/12/25 02:03:11 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/25 02:03:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:03:23 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:03:28 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:03:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:03:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:03:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:03:35 Testing delete operation on source bucket
2021/12/25 02:03:35 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:03:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/25 02:03:41 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:03:46 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:03:51 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:03:56 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/25 02:03:56 App: TestAdvancedDeleteOps got deployed
2021/12/25 02:03:56 Starting bucket ops to source bucket
2021/12/25 02:03:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:03:56 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:03:56 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:04:01 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:04:06 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:04:11 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:04:16 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/25 02:04:16 App: TestAdvancedDeleteOps got un-deployed
2021/12/25 02:04:16 No eventing-con process running
2021/12/25 02:04:16 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/25 02:04:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:04:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:04:25 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:04:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedDeleteOps (130.71s)
=== RUN   TestEnoentAdvancedGet
2021/12/25 02:04:30 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:04:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/25 02:04:36 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:04:42 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:04:47 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:04:52 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:04:52 App: TestEnoentAdvancedGet got deployed
2021/12/25 02:04:52 Starting bucket ops to source bucket
2021/12/25 02:04:57 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:05:02 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:05:02 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:05:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:05:02 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:05:07 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:05:12 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:05:17 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:05:22 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:05:22 App: TestEnoentAdvancedGet got un-deployed
2021/12/25 02:05:22 No eventing-con process running
2021/12/25 02:05:22 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/25 02:05:24 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:05:34 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:05:39 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:05:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:05:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:05:50 Waiting for dst bucket item count to get to: 0 curr count: 78
2021/12/25 02:05:55 Waiting for dst bucket item count to get to: 0 curr count: 78
2021/12/25 02:05:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:05:55 Testing get enoent operation on source bucket
2021/12/25 02:05:55 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:05:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/25 02:06:02 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:06:07 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:06:12 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:06:17 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/25 02:06:17 App: TestEnoentAdvancedGet got deployed
2021/12/25 02:06:17 Starting bucket ops to source bucket
2021/12/25 02:06:22 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/25 02:06:22 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/25 02:06:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:06:22 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:06:27 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:06:32 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:06:37 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:06:42 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/25 02:06:42 App: TestEnoentAdvancedGet got un-deployed
2021/12/25 02:06:42 No eventing-con process running
2021/12/25 02:06:42 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/25 02:06:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:06:54 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/25 02:06:59 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:06:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:07:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:07:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedGet (155.04s)
=== RUN   TestEnoentAdvancedReplace
2021/12/25 02:07:05 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:07:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/25 02:07:12 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:07:17 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:07:22 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:07:27 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:07:27 App: TestEnoentAdvancedReplace got deployed
2021/12/25 02:07:27 Starting bucket ops to source bucket
2021/12/25 02:07:32 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:07:32 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:07:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:07:32 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:07:37 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:07:42 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:07:47 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:07:52 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:07:52 App: TestEnoentAdvancedReplace got un-deployed
2021/12/25 02:07:52 No eventing-con process running
2021/12/25 02:07:52 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/25 02:07:54 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:08:04 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:08:09 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:08:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:08:10 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:08:20 Waiting for dst bucket item count to get to: 0 curr count: 79
2021/12/25 02:08:25 Waiting for dst bucket item count to get to: 0 curr count: 79
2021/12/25 02:08:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:08:25 Testing get enoent operation on source bucket
2021/12/25 02:08:25 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:08:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/25 02:08:32 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:08:37 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:08:42 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:08:47 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/25 02:08:47 App: TestEnoentAdvancedReplace got deployed
2021/12/25 02:08:47 Starting bucket ops to source bucket
2021/12/25 02:08:52 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/25 02:08:52 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/25 02:08:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:08:52 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:08:57 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:09:02 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:09:07 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:09:12 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/25 02:09:12 App: TestEnoentAdvancedReplace got un-deployed
2021/12/25 02:09:12 No eventing-con process running
2021/12/25 02:09:12 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/25 02:09:14 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:09:24 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/25 02:09:29 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:09:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:09:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:09:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedReplace (150.04s)
=== RUN   TestEnoentAdvancedDelete
2021/12/25 02:09:35 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:09:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/25 02:09:42 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:09:47 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:09:52 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:09:57 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:09:57 App: TestEnoentAdvancedDelete got deployed
2021/12/25 02:09:57 Starting bucket ops to source bucket
2021/12/25 02:10:02 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:10:02 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:10:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:10:02 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:10:07 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:10:12 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:10:17 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:10:22 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:10:22 App: TestEnoentAdvancedDelete got un-deployed
2021/12/25 02:10:22 No eventing-con process running
2021/12/25 02:10:22 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/25 02:10:24 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:10:34 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:10:39 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:10:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:10:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:10:50 Waiting for dst bucket item count to get to: 0 curr count: 80
2021/12/25 02:10:55 Waiting for dst bucket item count to get to: 0 curr count: 80
2021/12/25 02:10:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:10:55 Testing get enoent operation on source bucket
2021/12/25 02:10:55 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:10:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/25 02:11:02 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:11:07 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:11:12 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:11:17 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/25 02:11:17 App: TestEnoentAdvancedDelete got deployed
2021/12/25 02:11:17 Starting bucket ops to source bucket
2021/12/25 02:11:22 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/25 02:11:22 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/25 02:11:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:11:22 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:11:27 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:11:32 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:11:37 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:11:42 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/25 02:11:42 App: TestEnoentAdvancedDelete got un-deployed
2021/12/25 02:11:42 No eventing-con process running
2021/12/25 02:11:42 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/25 02:11:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:11:54 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/25 02:11:59 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:11:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:12:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:12:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedDelete (149.97s)
=== RUN   TestAdvancedInsertKeyExist
2021/12/25 02:12:05 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:12:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/25 02:12:12 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:12:17 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:12:22 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:12:27 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:12:27 App: TestAdvancedInsertKeyExist got deployed
2021/12/25 02:12:27 Starting bucket ops to source bucket
2021/12/25 02:12:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:12:27 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:12:27 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:12:32 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:12:37 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:12:42 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:12:47 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:12:47 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/25 02:12:47 No eventing-con process running
2021/12/25 02:12:47 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/25 02:12:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:12:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:12:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:13:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:13:01 Pumping document in the destination bucket
2021/12/25 02:13:01 Starting bucket ops to source bucket
2021/12/25 02:13:01 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:13:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/25 02:13:07 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:13:12 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:13:17 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:13:22 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/25 02:13:22 App: TestAdvancedInsertKeyExist got deployed
2021/12/25 02:13:22 Starting bucket ops to source bucket
2021/12/25 02:13:28 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:13:33 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:13:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:13:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:13:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:13:38 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:13:43 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:13:48 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:13:53 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/25 02:13:53 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/25 02:13:53 No eventing-con process running
2021/12/25 02:13:53 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/25 02:13:54 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:14:04 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:14:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:14:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:14:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertKeyExist (125.74s)
=== RUN   TestExipryGet
2021/12/25 02:14:11 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:14:12 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/25 02:14:17 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:14:22 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:14:27 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:14:32 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:14:32 App: TestExipryGet got deployed
2021/12/25 02:14:32 Starting bucket ops to source bucket
2021/12/25 02:15:18 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:15:23 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:15:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:15:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:15:23 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:15:28 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:15:33 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:15:38 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:15:43 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:15:43 App: TestExipryGet got un-deployed
2021/12/25 02:15:43 No eventing-con process running
2021/12/25 02:15:43 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/25 02:15:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:15:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:15:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:15:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:15:57 Pumping document in the destination bucket for expiry get
2021/12/25 02:15:57 Starting bucket ops to source bucket
2021/12/25 02:15:57 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:15:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/25 02:16:03 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:16:08 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:16:13 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:16:18 Waiting for app: TestExipryGet to get deployed
2021/12/25 02:16:18 App: TestExipryGet got deployed
2021/12/25 02:16:18 Starting bucket ops to source bucket
2021/12/25 02:17:04 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:17:09 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:17:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:17:09 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:17:09 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:17:14 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:17:19 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:17:24 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:17:29 Waiting for app: TestExipryGet to get un-deployed
2021/12/25 02:17:29 App: TestExipryGet got un-deployed
2021/12/25 02:17:29 No eventing-con process running
2021/12/25 02:17:29 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/25 02:17:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:17:40 Waiting for dst bucket item count to get to: 0 curr count: 79
2021/12/25 02:17:45 Waiting for dst bucket item count to get to: 0 curr count: 79
2021/12/25 02:17:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:17:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:17:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryGet (221.04s)
=== RUN   TestExipryInsert
2021/12/25 02:17:52 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:17:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/25 02:17:59 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:18:04 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:18:09 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:18:14 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:18:14 App: TestExipryInsert got deployed
2021/12/25 02:18:14 Starting bucket ops to source bucket
2021/12/25 02:18:59 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:19:04 Waiting for dst bucket item count to get to: 0 curr count: 75
2021/12/25 02:19:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:19:04 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:19:04 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:19:09 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:19:14 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:19:19 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:19:24 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:19:24 App: TestExipryInsert got un-deployed
2021/12/25 02:19:24 No eventing-con process running
2021/12/25 02:19:24 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/25 02:19:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:19:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:19:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:19:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:19:38 Testing on the destination bucket for expiry insert
2021/12/25 02:19:38 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:19:39 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/25 02:19:44 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:19:49 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:19:54 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:19:59 Waiting for app: TestExipryInsert to get deployed
2021/12/25 02:19:59 App: TestExipryInsert got deployed
2021/12/25 02:19:59 Starting bucket ops to source bucket
2021/12/25 02:20:44 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:20:49 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:20:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:20:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:20:49 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:20:54 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:20:59 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:21:04 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:21:09 Waiting for app: TestExipryInsert to get un-deployed
2021/12/25 02:21:09 App: TestExipryInsert got un-deployed
2021/12/25 02:21:09 No eventing-con process running
2021/12/25 02:21:09 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/25 02:21:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:21:21 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:21:26 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:21:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:21:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:21:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryInsert (220.66s)
=== RUN   TestExipryReplace
2021/12/25 02:21:33 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:21:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/25 02:21:39 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:21:44 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:21:49 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:21:54 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:21:54 App: TestExipryReplace got deployed
2021/12/25 02:21:54 Starting bucket ops to source bucket
2021/12/25 02:22:39 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:22:44 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:22:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:22:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:22:44 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:22:49 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:22:54 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:22:59 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:23:04 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:23:04 App: TestExipryReplace got un-deployed
2021/12/25 02:23:05 No eventing-con process running
2021/12/25 02:23:05 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/25 02:23:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:23:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:23:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:23:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:23:18 Testing on the destination bucket for expiry insert
2021/12/25 02:23:18 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:23:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/25 02:23:25 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:23:30 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:23:35 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:23:40 Waiting for app: TestExipryReplace to get deployed
2021/12/25 02:23:40 App: TestExipryReplace got deployed
2021/12/25 02:23:40 Starting bucket ops to source bucket
2021/12/25 02:24:25 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:24:30 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:24:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:24:30 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:24:30 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:24:35 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:24:40 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:24:45 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:24:50 Waiting for app: TestExipryReplace to get un-deployed
2021/12/25 02:24:50 App: TestExipryReplace got un-deployed
2021/12/25 02:24:50 No eventing-con process running
2021/12/25 02:24:50 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/25 02:24:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:25:02 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:25:07 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:25:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:25:08 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:25:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryReplace (220.56s)
=== RUN   TestExipryUpsert
2021/12/25 02:25:13 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:25:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/25 02:25:20 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:25:25 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:25:30 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:25:35 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:25:35 App: TestExipryUpsert got deployed
2021/12/25 02:25:35 Starting bucket ops to source bucket
2021/12/25 02:26:20 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:26:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:26:20 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:26:20 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:26:25 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:26:30 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:26:35 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:26:40 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:26:40 App: TestExipryUpsert got un-deployed
2021/12/25 02:26:40 No eventing-con process running
2021/12/25 02:26:40 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/25 02:26:42 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:26:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:26:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:26:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:26:54 Testing on the destination bucket for expiry insert
2021/12/25 02:26:54 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:26:55 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/25 02:27:00 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:27:05 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:27:10 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:27:15 Waiting for app: TestExipryUpsert to get deployed
2021/12/25 02:27:15 App: TestExipryUpsert got deployed
2021/12/25 02:27:15 Starting bucket ops to source bucket
2021/12/25 02:28:00 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:28:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:28:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:28:00 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:28:06 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:28:11 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:28:16 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:28:21 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/25 02:28:21 App: TestExipryUpsert got un-deployed
2021/12/25 02:28:21 No eventing-con process running
2021/12/25 02:28:21 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/25 02:28:22 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:28:32 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:28:37 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:28:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:28:39 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:28:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryUpsert (210.63s)
=== RUN   TestCasReplace
2021/12/25 02:28:44 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:28:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/25 02:28:50 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:28:55 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:29:00 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:29:05 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:29:05 App: TestCasReplace got deployed
2021/12/25 02:29:05 Starting bucket ops to source bucket
2021/12/25 02:29:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:29:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:29:06 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:29:11 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:29:16 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:29:21 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:29:26 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:29:26 App: TestCasReplace got un-deployed
2021/12/25 02:29:26 No eventing-con process running
2021/12/25 02:29:26 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/25 02:29:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:29:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:29:35 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:29:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:29:40 Testing on the destination bucket for upsert with cas
2021/12/25 02:29:40 Starting bucket ops to source bucket
2021/12/25 02:29:40 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:29:41 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/25 02:29:46 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:29:51 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:29:56 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:30:01 Waiting for app: TestCasReplace to get deployed
2021/12/25 02:30:01 App: TestCasReplace got deployed
2021/12/25 02:30:01 Starting bucket ops to source bucket
2021/12/25 02:30:06 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:30:11 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:30:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:30:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:30:11 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:30:16 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:30:21 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:30:26 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:30:31 Waiting for app: TestCasReplace to get un-deployed
2021/12/25 02:30:31 App: TestCasReplace got un-deployed
2021/12/25 02:30:31 No eventing-con process running
2021/12/25 02:30:32 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/25 02:30:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:30:43 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:30:48 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:30:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:30:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:30:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasReplace (130.93s)
=== RUN   TestCasDelete
2021/12/25 02:30:55 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:30:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/25 02:31:01 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:31:06 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:31:11 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:31:16 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:31:16 App: TestCasDelete got deployed
2021/12/25 02:31:16 Starting bucket ops to source bucket
2021/12/25 02:31:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:31:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:31:17 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:31:22 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:31:27 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:31:32 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:31:37 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:31:37 App: TestCasDelete got un-deployed
2021/12/25 02:31:37 No eventing-con process running
2021/12/25 02:31:37 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/25 02:31:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:31:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:31:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:31:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:31:51 Testing on the destination bucket for delete with cas
2021/12/25 02:31:51 Starting bucket ops to source bucket
2021/12/25 02:31:51 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:31:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/25 02:31:57 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:32:02 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:32:07 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:32:12 Waiting for app: TestCasDelete to get deployed
2021/12/25 02:32:12 App: TestCasDelete got deployed
2021/12/25 02:32:12 Starting bucket ops to source bucket
2021/12/25 02:32:17 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:32:22 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:32:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:32:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:32:22 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 02:32:27 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:32:32 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:32:37 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:32:42 Waiting for app: TestCasDelete to get un-deployed
2021/12/25 02:32:42 App: TestCasDelete got un-deployed
2021/12/25 02:32:42 No eventing-con process running
2021/12/25 02:32:42 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/25 02:32:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:32:54 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:32:59 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:32:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:33:01 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:33:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasDelete (130.86s)
=== RUN   TestCountersIncrement
2021/12/25 02:33:06 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 02:33:11 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 02:33:12 Removing node(s): [] from the cluster
2021/12/25 02:33:17 Rebalance progress: 100
2021/12/25 02:33:17 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:33:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/25 02:33:23 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:33:29 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:33:34 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:33:39 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:33:39 App: TestCountersIncrement got deployed
2021/12/25 02:33:39 Starting bucket ops to source bucket
2021/12/25 02:33:45 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/25 02:33:50 Waiting for dst bucket item count to get to: 1024 curr count: 379
2021/12/25 02:33:55 Waiting for dst bucket item count to get to: 1024 curr count: 379
2021/12/25 02:33:55 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/25 02:33:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:34:00 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:34:05 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:34:10 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:34:15 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:34:15 App: TestCountersIncrement got un-deployed
2021/12/25 02:34:15 No eventing-con process running
2021/12/25 02:34:15 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/25 02:34:17 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:34:27 Waiting for dst bucket item count to get to: 0 curr count: 1025
2021/12/25 02:34:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:34:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:34:38 Waiting for dst bucket item count to get to: 0 curr count: 881
2021/12/25 02:34:43 Waiting for dst bucket item count to get to: 0 curr count: 369
2021/12/25 02:34:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:34:43 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:34:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/25 02:34:50 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:34:55 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:35:00 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:35:05 Waiting for app: TestCountersIncrement to get deployed
2021/12/25 02:35:05 App: TestCountersIncrement got deployed
2021/12/25 02:35:05 Starting bucket ops to source bucket
2021/12/25 02:35:11 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/25 02:35:16 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/25 02:35:16 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/25 02:35:16 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:35:21 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:35:26 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:35:31 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:35:36 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/25 02:35:36 App: TestCountersIncrement got un-deployed
2021/12/25 02:35:36 No eventing-con process running
2021/12/25 02:35:36 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/25 02:35:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:35:48 Waiting for dst bucket item count to get to: 0 curr count: 1345
2021/12/25 02:35:53 Waiting for dst bucket item count to get to: 0 curr count: 321
2021/12/25 02:35:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:35:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:36:05 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/25 02:36:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:36:05 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/25 02:36:10 Rebalance progress: 100
--- PASS: TestCountersIncrement (183.92s)
=== RUN   TestCountersDecrement
2021/12/25 02:36:10 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 02:36:10 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 02:36:10 Removing node(s): [] from the cluster
2021/12/25 02:36:15 Rebalance progress: 100
2021/12/25 02:36:15 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:36:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/25 02:36:22 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:36:27 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:36:32 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:36:37 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:36:37 App: TestCountersDecrement got deployed
2021/12/25 02:36:37 Starting bucket ops to source bucket
2021/12/25 02:36:43 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/25 02:36:48 Waiting for dst bucket item count to get to: 1024 curr count: 690
2021/12/25 02:36:53 Waiting for dst bucket item count to get to: 1024 curr count: 690
2021/12/25 02:36:53 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/25 02:36:53 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:36:58 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:37:03 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:37:08 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:37:13 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:37:13 App: TestCountersDecrement got un-deployed
2021/12/25 02:37:13 No eventing-con process running
2021/12/25 02:37:13 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/25 02:37:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:37:25 Waiting for dst bucket item count to get to: 0 curr count: 1025
2021/12/25 02:37:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:37:27 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:37:37 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/25 02:37:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:37:37 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:37:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/25 02:37:43 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:37:48 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:37:53 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:37:58 Waiting for app: TestCountersDecrement to get deployed
2021/12/25 02:37:58 App: TestCountersDecrement got deployed
2021/12/25 02:37:58 Starting bucket ops to source bucket
2021/12/25 02:38:04 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/25 02:38:09 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/25 02:38:14 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/25 02:38:14 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/25 02:38:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:38:19 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:38:24 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:38:29 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:38:34 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/25 02:38:34 App: TestCountersDecrement got un-deployed
2021/12/25 02:38:34 No eventing-con process running
2021/12/25 02:38:34 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/25 02:38:36 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:38:46 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/25 02:38:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:38:48 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:38:58 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/25 02:38:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:38:58 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/25 02:39:03 Rebalance progress: 100
--- PASS: TestCountersDecrement (173.22s)
=== RUN   TestCacheGet
2021/12/25 02:39:03 Deploying app: TestCacheGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:39:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet stored in temp store"
}
2021/12/25 02:39:09 Waiting for app: TestCacheGet to get deployed
2021/12/25 02:39:14 Waiting for app: TestCacheGet to get deployed
2021/12/25 02:39:19 Waiting for app: TestCacheGet to get deployed
2021/12/25 02:39:24 Waiting for app: TestCacheGet to get deployed
2021/12/25 02:39:24 App: TestCacheGet got deployed
2021/12/25 02:39:24 Starting bucket ops to source bucket
2021/12/25 02:39:29 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/25 02:39:34 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/25 02:39:39 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/25 02:39:44 Waiting for dst bucket item count to get to: 20 curr count: 6
2021/12/25 02:39:49 Waiting for dst bucket item count to get to: 20 curr count: 14
2021/12/25 02:39:54 Waiting for dst bucket item count to get to: 20 curr count: 18
2021/12/25 02:39:54 src & dst bucket item count matched up. src bucket count: 20 dst bucket count: 20
2021/12/25 02:39:54 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:40:00 Waiting for app: TestCacheGet to get un-deployed
2021/12/25 02:40:05 Waiting for app: TestCacheGet to get un-deployed
2021/12/25 02:40:10 Waiting for app: TestCacheGet to get un-deployed
2021/12/25 02:40:15 Waiting for app: TestCacheGet to get un-deployed
2021/12/25 02:40:15 App: TestCacheGet got un-deployed
2021/12/25 02:40:15 No eventing-con process running
2021/12/25 02:40:15 Function: TestCacheGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet deleting in background"
}
2021/12/25 02:40:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:40:26 Waiting for dst bucket item count to get to: 0 curr count: 20
2021/12/25 02:40:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:40:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:40:38 Waiting for dst bucket item count to get to: 0 curr count: 16
2021/12/25 02:40:43 Waiting for dst bucket item count to get to: 0 curr count: 6
2021/12/25 02:40:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheGet (100.32s)
=== RUN   TestCacheRYOW
2021/12/25 02:40:43 Deploying app: TestCacheRYOW. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:40:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW stored in temp store"
}
2021/12/25 02:40:50 Waiting for app: TestCacheRYOW to get deployed
2021/12/25 02:40:55 Waiting for app: TestCacheRYOW to get deployed
2021/12/25 02:41:00 Waiting for app: TestCacheRYOW to get deployed
2021/12/25 02:41:05 Waiting for app: TestCacheRYOW to get deployed
2021/12/25 02:41:05 App: TestCacheRYOW got deployed
2021/12/25 02:41:05 Starting bucket ops to source bucket
2021/12/25 02:41:10 Waiting for dst bucket item count to get to: 50 curr count: 0
2021/12/25 02:41:15 Waiting for dst bucket item count to get to: 50 curr count: 25
2021/12/25 02:41:15 src & dst bucket item count matched up. src bucket count: 50 dst bucket count: 50
2021/12/25 02:41:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:41:20 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/25 02:41:25 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/25 02:41:30 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/25 02:41:35 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/25 02:41:35 App: TestCacheRYOW got un-deployed
2021/12/25 02:41:35 No eventing-con process running
2021/12/25 02:41:35 Function: TestCacheRYOW delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW deleting in background"
}
2021/12/25 02:41:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:41:47 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:41:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:41:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:41:59 Waiting for dst bucket item count to get to: 0 curr count: 43
2021/12/25 02:42:04 Waiting for dst bucket item count to get to: 0 curr count: 18
2021/12/25 02:42:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheRYOW (80.29s)
=== RUN   TestCacheExpiry
2021/12/25 02:42:04 Deploying app: TestCacheExpiry. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:42:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry stored in temp store"
}
2021/12/25 02:42:10 Waiting for app: TestCacheExpiry to get deployed
2021/12/25 02:42:15 Waiting for app: TestCacheExpiry to get deployed
2021/12/25 02:42:20 Waiting for app: TestCacheExpiry to get deployed
2021/12/25 02:42:25 Waiting for app: TestCacheExpiry to get deployed
2021/12/25 02:42:25 App: TestCacheExpiry got deployed
2021/12/25 02:42:25 Starting bucket ops to source bucket
2021/12/25 02:42:30 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:42:35 Waiting for dst bucket item count to get to: 100 curr count: 14
2021/12/25 02:42:40 Waiting for dst bucket item count to get to: 100 curr count: 37
2021/12/25 02:42:45 Waiting for dst bucket item count to get to: 100 curr count: 99
2021/12/25 02:42:50 Waiting for dst bucket item count to get to: 100 curr count: 99
2021/12/25 02:42:50 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:42:50 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:42:55 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/25 02:43:00 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/25 02:43:05 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/25 02:43:10 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/25 02:43:10 App: TestCacheExpiry got un-deployed
2021/12/25 02:43:10 No eventing-con process running
2021/12/25 02:43:10 Function: TestCacheExpiry delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry deleting in background"
}
2021/12/25 02:43:12 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:43:22 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:43:27 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:43:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:43:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:43:39 Waiting for dst bucket item count to get to: 0 curr count: 99
2021/12/25 02:43:44 Waiting for dst bucket item count to get to: 0 curr count: 49
2021/12/25 02:43:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiry (100.33s)
=== RUN   TestCacheExpiryCustom
2021/12/25 02:43:44 Deploying app: TestCacheExpiryCustom. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:43:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom stored in temp store"
}
2021/12/25 02:43:50 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/25 02:43:55 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/25 02:44:00 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/25 02:44:05 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/25 02:44:05 App: TestCacheExpiryCustom got deployed
2021/12/25 02:44:05 Starting bucket ops to source bucket
2021/12/25 02:44:10 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:44:15 Waiting for dst bucket item count to get to: 100 curr count: 20
2021/12/25 02:44:20 Waiting for dst bucket item count to get to: 100 curr count: 20
2021/12/25 02:44:25 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/25 02:44:30 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/25 02:44:30 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:44:30 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:44:36 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/25 02:44:41 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/25 02:44:46 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/25 02:44:51 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/25 02:44:51 App: TestCacheExpiryCustom got un-deployed
2021/12/25 02:44:51 No eventing-con process running
2021/12/25 02:44:51 Function: TestCacheExpiryCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom deleting in background"
}
2021/12/25 02:44:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:45:02 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:45:07 Waiting for dst bucket item count to get to: 0 curr count: 25
2021/12/25 02:45:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:45:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:45:19 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:45:24 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:45:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiryCustom (100.33s)
=== RUN   TestCacheOverflowCustom
2021/12/25 02:45:24 Deploying app: TestCacheOverflowCustom. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:45:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom stored in temp store"
}
2021/12/25 02:45:30 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/25 02:45:36 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/25 02:45:41 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/25 02:45:46 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/25 02:45:46 App: TestCacheOverflowCustom got deployed
2021/12/25 02:45:46 Starting bucket ops to source bucket
2021/12/25 02:45:51 Waiting for dst bucket item count to get to: 35 curr count: 0
2021/12/25 02:45:56 Waiting for dst bucket item count to get to: 35 curr count: 13
2021/12/25 02:46:01 Waiting for dst bucket item count to get to: 35 curr count: 13
2021/12/25 02:46:06 Waiting for dst bucket item count to get to: 35 curr count: 27
2021/12/25 02:46:11 Waiting for dst bucket item count to get to: 35 curr count: 27
2021/12/25 02:46:11 src & dst bucket item count matched up. src bucket count: 35 dst bucket count: 35
2021/12/25 02:46:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:46:16 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/25 02:46:21 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/25 02:46:26 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/25 02:46:31 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/25 02:46:31 App: TestCacheOverflowCustom got un-deployed
2021/12/25 02:46:31 No eventing-con process running
2021/12/25 02:46:31 Function: TestCacheOverflowCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom deleting in background"
}
2021/12/25 02:46:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:46:43 Waiting for dst bucket item count to get to: 0 curr count: 5
2021/12/25 02:46:48 Waiting for dst bucket item count to get to: 0 curr count: 3
2021/12/25 02:46:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:46:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:47:00 Waiting for dst bucket item count to get to: 0 curr count: 17
2021/12/25 02:47:05 Waiting for dst bucket item count to get to: 0 curr count: 17
2021/12/25 02:47:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheOverflowCustom (100.32s)
=== RUN   TestStrictMode
2021/12/25 02:47:05 Deploying app: TestStrictMode. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:47:06 Context: Post to main store, response code: 422 dump: {
 "name": "ERR_HANDLER_COMPILATION",
 "code": 27,
 "description": {
  "area": "handlerCode",
  "column_number": 63,
  "compile_success": false,
  "description": "        destination[meta.id] = 00000000000000000000000000000101;\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nSyntaxError: Octal literals are not allowed in strict mode.\n",
  "index": 132,
  "language": "JavaScript",
  "line_number": 4
 }
}
--- PASS: TestStrictMode (1.07s)
=== RUN   TestDataTypes
2021/12/25 02:47:06 Deploying app: TestDataTypes. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:47:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes stored in temp store"
}
2021/12/25 02:47:12 Waiting for app: TestDataTypes to get deployed
2021/12/25 02:47:17 Waiting for app: TestDataTypes to get deployed
2021/12/25 02:47:22 Waiting for app: TestDataTypes to get deployed
2021/12/25 02:47:27 Waiting for app: TestDataTypes to get deployed
2021/12/25 02:47:27 App: TestDataTypes got deployed
2021/12/25 02:47:27 Starting bucket ops to source bucket
2021/12/25 02:47:32 Waiting for dst bucket item count to get to: 18 curr count: 0
2021/12/25 02:47:32 src & dst bucket item count matched up. src bucket count: 18 dst bucket count: 18
2021/12/25 02:47:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:47:37 Waiting for app: TestDataTypes to get un-deployed
2021/12/25 02:47:42 Waiting for app: TestDataTypes to get un-deployed
2021/12/25 02:47:47 Waiting for app: TestDataTypes to get un-deployed
2021/12/25 02:47:52 Waiting for app: TestDataTypes to get un-deployed
2021/12/25 02:47:52 App: TestDataTypes got un-deployed
2021/12/25 02:47:52 No eventing-con process running
2021/12/25 02:47:52 Function: TestDataTypes delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes deleting in background"
}
2021/12/25 02:47:54 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:48:04 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/25 02:48:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:48:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:48:16 Waiting for dst bucket item count to get to: 0 curr count: 18
2021/12/25 02:48:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDataTypes (70.09s)
=== RUN   TestEnoentGet
2021/12/25 02:48:16 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:48:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/25 02:48:22 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:48:27 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:48:32 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:48:37 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:48:37 App: TestEnoentGet got deployed
2021/12/25 02:48:37 Starting bucket ops to source bucket
2021/12/25 02:48:42 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:48:42 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:48:42 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:48:47 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:48:52 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:48:57 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:49:02 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:49:02 App: TestEnoentGet got un-deployed
2021/12/25 02:49:02 No eventing-con process running
2021/12/25 02:49:02 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/25 02:49:04 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:49:14 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:49:19 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:49:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:49:21 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:49:31 Waiting for dst bucket item count to get to: 0 curr count: 99
2021/12/25 02:49:36 Waiting for dst bucket item count to get to: 0 curr count: 99
2021/12/25 02:49:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:49:36 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:49:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/25 02:49:42 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:49:47 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:49:52 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:49:57 Waiting for app: TestEnoentGet to get deployed
2021/12/25 02:49:57 App: TestEnoentGet got deployed
2021/12/25 02:49:57 Starting bucket ops to source bucket
2021/12/25 02:50:02 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:50:07 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:50:12 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:50:12 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:50:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:50:18 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:50:23 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:50:28 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:50:33 Waiting for app: TestEnoentGet to get un-deployed
2021/12/25 02:50:33 App: TestEnoentGet got un-deployed
2021/12/25 02:50:33 No eventing-con process running
2021/12/25 02:50:33 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/25 02:50:34 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:50:44 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:50:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:50:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:50:56 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:50:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentGet (160.45s)
=== RUN   TestEnoentDelete
2021/12/25 02:50:56 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:50:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/25 02:51:02 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:51:07 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:51:12 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:51:17 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:51:17 App: TestEnoentDelete got deployed
2021/12/25 02:51:17 Starting bucket ops to source bucket
2021/12/25 02:51:23 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:51:28 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:51:33 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:51:33 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:51:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:51:38 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:51:43 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:51:48 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:51:53 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:51:53 App: TestEnoentDelete got un-deployed
2021/12/25 02:51:53 No eventing-con process running
2021/12/25 02:51:53 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/25 02:51:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:52:05 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:52:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:52:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:52:16 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:52:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:52:16 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:52:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/25 02:52:23 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:52:28 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:52:33 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:52:38 Waiting for app: TestEnoentDelete to get deployed
2021/12/25 02:52:38 App: TestEnoentDelete got deployed
2021/12/25 02:52:38 Starting bucket ops to source bucket
2021/12/25 02:52:43 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:52:48 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:52:53 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 02:52:53 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:52:53 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:52:58 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:53:03 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:53:08 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:53:13 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/25 02:53:13 App: TestEnoentDelete got un-deployed
2021/12/25 02:53:13 No eventing-con process running
2021/12/25 02:53:13 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/25 02:53:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:53:25 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:53:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:53:27 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:53:37 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:53:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentDelete (160.52s)
=== RUN   TestError
2021/12/25 02:53:47 Waiting for app: TestError to get un-deployed
2021/12/25 02:53:47 App: TestError got un-deployed
2021/12/25 02:53:47 No eventing-con process running
2021/12/25 02:53:47 Function: TestError delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestError not found"
}
2021/12/25 02:53:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:53:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:53:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:54:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:54:00 Deploying app: TestError. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:54:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError stored in temp store"
}
2021/12/25 02:54:06 Waiting for app: TestError to get deployed
2021/12/25 02:54:12 Waiting for app: TestError to get deployed
2021/12/25 02:54:17 Waiting for app: TestError to get deployed
2021/12/25 02:54:22 Waiting for app: TestError to get deployed
2021/12/25 02:54:22 App: TestError got deployed
2021/12/25 02:54:22 Starting bucket ops to source bucket
2021/12/25 02:54:27 Waiting for dst bucket item count to get to: 300 curr count: 0
2021/12/25 02:54:32 Waiting for dst bucket item count to get to: 300 curr count: 0
2021/12/25 02:54:32 src & dst bucket item count matched up. src bucket count: 300 dst bucket count: 300
2021/12/25 02:54:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:54:37 Waiting for app: TestError to get un-deployed
2021/12/25 02:54:42 Waiting for app: TestError to get un-deployed
2021/12/25 02:54:47 Waiting for app: TestError to get un-deployed
2021/12/25 02:54:52 Waiting for app: TestError to get un-deployed
2021/12/25 02:54:52 App: TestError got un-deployed
2021/12/25 02:54:52 No eventing-con process running
2021/12/25 02:54:52 Function: TestError delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError deleting in background"
}
2021/12/25 02:54:54 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:55:04 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:55:09 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:55:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:55:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:55:21 Waiting for dst bucket item count to get to: 0 curr count: 262
2021/12/25 02:55:26 Waiting for dst bucket item count to get to: 0 curr count: 262
2021/12/25 02:55:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestError (108.95s)
=== RUN   TestCRLF
2021/12/25 02:55:36 Waiting for app: TestCRLF to get un-deployed
2021/12/25 02:55:36 App: TestCRLF got un-deployed
2021/12/25 02:55:36 No eventing-con process running
2021/12/25 02:55:36 Function: TestCRLF delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCRLF not found"
}
2021/12/25 02:55:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:55:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:55:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:55:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:55:49 Deploying app: TestCRLF. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:55:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF stored in temp store"
}
2021/12/25 02:55:55 Waiting for app: TestCRLF to get deployed
2021/12/25 02:56:00 Waiting for app: TestCRLF to get deployed
2021/12/25 02:56:05 Waiting for app: TestCRLF to get deployed
2021/12/25 02:56:11 Waiting for app: TestCRLF to get deployed
2021/12/25 02:56:11 App: TestCRLF got deployed
2021/12/25 02:56:11 Starting bucket ops to source bucket
2021/12/25 02:56:16 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:56:21 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 02:56:21 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 02:56:21 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 02:56:26 Waiting for app: TestCRLF to get un-deployed
2021/12/25 02:56:31 Waiting for app: TestCRLF to get un-deployed
2021/12/25 02:56:36 Waiting for app: TestCRLF to get un-deployed
2021/12/25 02:56:41 Waiting for app: TestCRLF to get un-deployed
2021/12/25 02:56:41 App: TestCRLF got un-deployed
2021/12/25 02:56:41 No eventing-con process running
2021/12/25 02:56:41 Function: TestCRLF delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF deleting in background"
}
2021/12/25 02:56:43 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:56:53 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 02:56:58 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/25 02:56:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:57:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:57:10 Waiting for dst bucket item count to get to: 0 curr count: 47
2021/12/25 02:57:15 Waiting for dst bucket item count to get to: 0 curr count: 47
2021/12/25 02:57:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCRLF (108.94s)
=== RUN   TestImportExport
2021/12/25 02:57:25 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:57:25 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/25 02:57:25 No eventing-con process running
2021/12/25 02:57:25 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_n1ql_insert_on_update not found"
}
2021/12/25 02:57:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:57:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:57:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:57:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:57:43 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:57:43 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/25 02:57:43 No eventing-con process running
2021/12/25 02:57:43 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_bucket_op_on_update not found"
}
2021/12/25 02:57:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:57:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:57:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:57:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:57:57 Deploying app: TestImportExport_n1ql_insert_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:57:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update stored in temp store"
}
2021/12/25 02:57:58 Deploying app: TestImportExport_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 02:57:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update stored in temp store"
}
2021/12/25 02:58:04 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/25 02:58:09 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/25 02:58:14 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/25 02:58:19 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/25 02:58:19 App: TestImportExport_n1ql_insert_on_update got deployed
2021/12/25 02:58:24 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/25 02:58:29 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/25 02:58:34 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/25 02:58:34 App: TestImportExport_bucket_op_on_update got deployed
2021/12/25 02:58:39 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:58:44 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:58:49 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:58:54 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:58:55 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/25 02:58:55 No eventing-con process running
2021/12/25 02:58:55 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/25 02:59:00 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:59:05 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:59:10 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:59:15 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:59:15 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/25 02:59:15 No eventing-con process running
2021/12/25 02:59:15 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/25 02:59:32 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/25 02:59:32 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/25 02:59:32 No eventing-con process running
2021/12/25 02:59:32 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/25 02:59:34 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:59:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:59:41 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:59:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:59:51 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/25 02:59:51 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/25 02:59:51 No eventing-con process running
2021/12/25 02:59:51 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/25 02:59:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 02:59:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 02:59:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:00:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestImportExport (169.76s)
=== RUN   TestDeployUndeployLoopNonDefaultSettings
2021/12/25 03:00:14 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:00:14 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:00:14 No eventing-con process running
2021/12/25 03:00:14 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopNonDefaultSettings not found"
}
2021/12/25 03:00:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:00:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:00:23 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:00:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:00:28 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:00:29 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:00:29 Starting bucket ops to source bucket
2021/12/25 03:00:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:00:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:00:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:00:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:00:57 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/25 03:00:57 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:01:02 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/25 03:01:02 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/25 03:01:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:01:02 Undeploying app: bucket_op_on_update
2021/12/25 03:01:07 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:01:12 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:01:17 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:01:22 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:01:22 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:01:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:01:22 No eventing-con process running
2021/12/25 03:01:24 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:01:31 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:02:06 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:02:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:02:07 Starting bucket ops to source bucket
2021/12/25 03:02:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:02:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:02:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:02:30 Waiting for dst bucket item count to get to: 5000 curr count: 291
2021/12/25 03:02:35 Waiting for dst bucket item count to get to: 5000 curr count: 291
2021/12/25 03:02:35 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:02:40 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/25 03:02:40 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/25 03:02:40 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:02:40 Undeploying app: bucket_op_on_update
2021/12/25 03:02:46 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:02:51 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:02:56 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:03:01 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:03:01 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:03:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:03:01 No eventing-con process running
2021/12/25 03:03:02 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:03:10 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:03:45 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:03:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:03:46 Starting bucket ops to source bucket
2021/12/25 03:03:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:03:59 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:04:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:04:09 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/25 03:04:14 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/25 03:04:14 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:04:19 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/25 03:04:19 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/25 03:04:19 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:04:19 Undeploying app: bucket_op_on_update
2021/12/25 03:04:24 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:04:29 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:04:34 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:04:39 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:04:39 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:04:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:04:39 No eventing-con process running
2021/12/25 03:04:41 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:04:48 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:05:23 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:05:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:05:24 Starting bucket ops to source bucket
2021/12/25 03:05:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:05:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:05:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:05:47 Waiting for dst bucket item count to get to: 5000 curr count: 1913
2021/12/25 03:05:52 Waiting for dst bucket item count to get to: 5000 curr count: 4413
2021/12/25 03:05:52 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:05:57 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/25 03:05:57 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/25 03:05:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:05:57 Undeploying app: bucket_op_on_update
2021/12/25 03:06:02 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:06:07 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:06:12 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:06:17 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:06:17 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:06:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:06:17 No eventing-con process running
2021/12/25 03:06:19 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:06:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:07:01 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:07:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:07:02 Starting bucket ops to source bucket
2021/12/25 03:07:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:07:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:07:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:07:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:07:25 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:07:30 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/25 03:07:30 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/25 03:07:30 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:07:30 Undeploying app: bucket_op_on_update
2021/12/25 03:07:35 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:07:41 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:07:46 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:07:51 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/25 03:07:51 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/25 03:07:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:07:51 No eventing-con process running
2021/12/25 03:07:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:08:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:08:35 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings deleting in background"
}
--- PASS: TestDeployUndeployLoopNonDefaultSettings (510.25s)
=== RUN   TestOnUpdateN1QLOp
2021/12/25 03:08:45 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/25 03:08:45 App: TestOnUpdateN1QLOp got un-deployed
2021/12/25 03:08:45 No eventing-con process running
2021/12/25 03:08:45 Function: TestOnUpdateN1QLOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateN1QLOp not found"
}
2021/12/25 03:08:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:08:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:08:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:08:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:08:58 Deploying app: TestOnUpdateN1QLOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:08:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp stored in temp store"
}
2021/12/25 03:08:59 Starting bucket ops to source bucket
2021/12/25 03:09:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:09:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:09:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:09:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:09:27 Waiting for dst bucket item count to get to: 5000 curr count: 2535
2021/12/25 03:09:27 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:09:27 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:09:32 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/25 03:09:37 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/25 03:09:42 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/25 03:09:47 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/25 03:09:47 App: TestOnUpdateN1QLOp got un-deployed
2021/12/25 03:09:47 No eventing-con process running
2021/12/25 03:09:48 Function: TestOnUpdateN1QLOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp deleting in background"
}
2021/12/25 03:09:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:09:59 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:10:04 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:10:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:10:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:10:16 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:10:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateN1QLOp (101.48s)
=== RUN   TestOnUpdateBucketOpDefaultSettings
2021/12/25 03:10:26 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/25 03:10:26 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/25 03:10:26 No eventing-con process running
2021/12/25 03:10:26 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings not found"
}
2021/12/25 03:10:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:10:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:10:35 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:10:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:10:40 Deploying app: TestOnUpdateBucketOpDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:10:41 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings stored in temp store"
}
2021/12/25 03:10:41 Starting bucket ops to source bucket
2021/12/25 03:10:49 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:10:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:10:59 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:11:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:11:04 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:11:04 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:11:09 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/25 03:11:14 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/25 03:11:19 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/25 03:11:24 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/25 03:11:24 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/25 03:11:24 No eventing-con process running
2021/12/25 03:11:24 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings deleting in background"
}
2021/12/25 03:11:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:11:36 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:11:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:11:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:11:48 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:11:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings (91.53s)
=== RUN   TestOnUpdateBucketOpNonDefaultSettings
2021/12/25 03:11:58 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 03:11:58 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/25 03:11:58 No eventing-con process running
2021/12/25 03:11:58 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/25 03:11:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:12:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:12:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:12:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:12:11 Deploying app: TestOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:12:12 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/25 03:12:12 Starting bucket ops to source bucket
2021/12/25 03:12:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:12:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:12:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:12:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:12:35 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:12:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:12:41 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 03:12:46 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 03:12:51 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 03:12:56 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 03:12:56 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/25 03:12:56 No eventing-con process running
2021/12/25 03:12:56 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/25 03:12:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:13:08 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:13:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:13:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:13:19 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:13:24 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:13:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpNonDefaultSettings (96.64s)
=== RUN   TestOnUpdateBucketOpDefaultSettings10K
2021/12/25 03:13:34 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/25 03:13:34 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/25 03:13:34 No eventing-con process running
2021/12/25 03:13:34 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings10K not found"
}
2021/12/25 03:13:36 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:13:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:13:43 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:13:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:13:48 Deploying app: TestOnUpdateBucketOpDefaultSettings10K. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:13:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K stored in temp store"
}
2021/12/25 03:13:49 Starting bucket ops to source bucket
2021/12/25 03:13:57 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:14:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:14:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:14:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:14:17 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/25 03:14:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:14:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:14:22 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/25 03:14:27 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/25 03:14:32 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/25 03:14:37 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/25 03:14:37 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/25 03:14:37 No eventing-con process running
2021/12/25 03:14:37 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K deleting in background"
}
2021/12/25 03:14:39 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:14:49 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:14:54 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:14:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:14:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:15:06 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:15:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings10K (101.59s)
=== RUN   TestOnUpdateBucketOpDefaultSettings100K
2021/12/25 03:15:16 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/25 03:15:16 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/25 03:15:16 No eventing-con process running
2021/12/25 03:15:16 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings100K not found"
}
2021/12/25 03:15:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:15:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:15:24 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:15:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:15:29 Deploying app: TestOnUpdateBucketOpDefaultSettings100K. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:15:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K stored in temp store"
}
2021/12/25 03:15:31 Starting bucket ops to source bucket
2021/12/25 03:15:39 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:15:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:15:49 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:15:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:15:54 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:15:54 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:15:59 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/25 03:16:04 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/25 03:16:09 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/25 03:16:14 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/25 03:16:14 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/25 03:16:14 No eventing-con process running
2021/12/25 03:16:14 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K deleting in background"
}
2021/12/25 03:16:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:16:26 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:16:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:16:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:16:38 Waiting for dst bucket item count to get to: 0 curr count: 3436
2021/12/25 03:16:43 Waiting for dst bucket item count to get to: 0 curr count: 936
2021/12/25 03:16:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings100K (97.15s)
=== RUN   TestOnDeleteBucketOp
2021/12/25 03:16:53 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/25 03:16:53 App: TestOnDeleteBucketOp got un-deployed
2021/12/25 03:16:53 No eventing-con process running
2021/12/25 03:16:53 Function: TestOnDeleteBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp not found"
}
2021/12/25 03:16:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:17:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:17:02 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:17:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:17:07 Deploying app: TestOnDeleteBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:17:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp stored in temp store"
}
2021/12/25 03:17:08 Starting bucket ops to source bucket
2021/12/25 03:17:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:17:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:17:29 Waiting for dst bucket item count to get to: 5000 curr count: 222
2021/12/25 03:17:34 Waiting for dst bucket item count to get to: 5000 curr count: 222
2021/12/25 03:17:34 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:17:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:17:39 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/25 03:17:44 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/25 03:17:49 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/25 03:17:54 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/25 03:17:54 App: TestOnDeleteBucketOp got un-deployed
2021/12/25 03:17:54 No eventing-con process running
2021/12/25 03:17:54 Function: TestOnDeleteBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp deleting in background"
}
2021/12/25 03:17:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:18:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:18:03 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:18:13 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:18:18 Waiting for dst bucket item count to get to: 0 curr count: 2496
2021/12/25 03:18:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp (94.95s)
=== RUN   TestOnDeleteBucketOp5K
2021/12/25 03:18:28 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/25 03:18:28 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/25 03:18:28 No eventing-con process running
2021/12/25 03:18:28 Function: TestOnDeleteBucketOp5K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp5K not found"
}
2021/12/25 03:18:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:18:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:18:36 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:18:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:18:41 Deploying app: TestOnDeleteBucketOp5K. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:18:43 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K stored in temp store"
}
2021/12/25 03:18:43 Starting bucket ops to source bucket
2021/12/25 03:18:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:18:58 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:19:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:19:03 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:19:04 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:19:09 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/25 03:19:14 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/25 03:19:19 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/25 03:19:24 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/25 03:19:24 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/25 03:19:24 No eventing-con process running
2021/12/25 03:19:24 Function: TestOnDeleteBucketOp5K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K deleting in background"
}
2021/12/25 03:19:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:19:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:19:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:19:43 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:19:48 Waiting for dst bucket item count to get to: 0 curr count: 2496
2021/12/25 03:19:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp5K (89.88s)
=== RUN   TestTimerBucketOp
2021/12/25 03:19:58 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/25 03:19:58 App: TestTimerBucketOp got un-deployed
2021/12/25 03:19:58 No eventing-con process running
2021/12/25 03:19:58 Function: TestTimerBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerBucketOp not found"
}
2021/12/25 03:20:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:20:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:20:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:20:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:20:11 Deploying app: TestTimerBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:20:12 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp stored in temp store"
}
2021/12/25 03:20:12 Starting bucket ops to source bucket
2021/12/25 03:20:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:20:50 Waiting for dst bucket item count to get to: 5000 curr count: 3287
2021/12/25 03:20:55 Waiting for dst bucket item count to get to: 5000 curr count: 3287
2021/12/25 03:20:55 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:20:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:21:01 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/25 03:21:06 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/25 03:21:11 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/25 03:21:16 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/25 03:21:16 App: TestTimerBucketOp got un-deployed
2021/12/25 03:21:16 No eventing-con process running
2021/12/25 03:21:16 Function: TestTimerBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp deleting in background"
}
2021/12/25 03:21:17 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:21:27 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:21:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:21:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:21:39 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:21:44 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:21:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerBucketOp (116.40s)
=== RUN   TestTimerInPastBucketOp
2021/12/25 03:21:54 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/25 03:21:54 App: TestTimerInPastBucketOp got un-deployed
2021/12/25 03:21:54 No eventing-con process running
2021/12/25 03:21:54 Function: TestTimerInPastBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerInPastBucketOp not found"
}
2021/12/25 03:21:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:22:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:22:03 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:22:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:22:08 Deploying app: TestTimerInPastBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:22:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp stored in temp store"
}
2021/12/25 03:22:09 Starting bucket ops to source bucket
2021/12/25 03:22:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:27 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:22:47 Waiting for dst bucket item count to get to: 5000 curr count: 1953
2021/12/25 03:22:52 Waiting for dst bucket item count to get to: 5000 curr count: 4453
2021/12/25 03:22:52 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:22:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:22:57 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/25 03:23:02 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/25 03:23:07 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/25 03:23:12 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/25 03:23:12 App: TestTimerInPastBucketOp got un-deployed
2021/12/25 03:23:12 No eventing-con process running
2021/12/25 03:23:12 Function: TestTimerInPastBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp deleting in background"
}
2021/12/25 03:23:14 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:23:24 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:23:29 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 03:23:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:23:31 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:23:41 Waiting for dst bucket item count to get to: 0 curr count: 4477
2021/12/25 03:23:46 Waiting for dst bucket item count to get to: 0 curr count: 4477
2021/12/25 03:23:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerInPastBucketOp (121.44s)
=== RUN   TestDeployUndeployLoopTimer
2021/12/25 03:23:56 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:23:56 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:23:56 No eventing-con process running
2021/12/25 03:23:56 Function: TestDeployUndeployLoopTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopTimer not found"
}
2021/12/25 03:23:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:24:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:24:04 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:24:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:24:09 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:24:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/25 03:24:10 Starting bucket ops to source bucket
2021/12/25 03:24:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:28 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:38 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:24:48 Waiting for dst bucket item count to get to: 5000 curr count: 4726
2021/12/25 03:24:53 Waiting for dst bucket item count to get to: 5000 curr count: 4726
2021/12/25 03:24:53 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:24:58 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/25 03:24:58 App: TestDeployUndeployLoopTimer got deployed
2021/12/25 03:24:58 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:24:58 Undeploying app: bucket_op_with_timer
2021/12/25 03:25:04 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:25:09 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:25:14 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:25:19 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:25:19 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:25:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:25:19 No eventing-con process running
2021/12/25 03:25:20 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:25:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:26:03 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:26:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/25 03:26:04 Starting bucket ops to source bucket
2021/12/25 03:26:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:26:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:26:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:26:27 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:26:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:26:37 Waiting for dst bucket item count to get to: 5000 curr count: 301
2021/12/25 03:26:42 Waiting for dst bucket item count to get to: 5000 curr count: 549
2021/12/25 03:26:47 Waiting for dst bucket item count to get to: 5000 curr count: 2748
2021/12/25 03:26:47 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:26:52 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/25 03:26:52 App: TestDeployUndeployLoopTimer got deployed
2021/12/25 03:26:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:26:52 Undeploying app: bucket_op_with_timer
2021/12/25 03:26:57 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:27:02 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:27:07 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:27:12 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:27:12 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:27:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:27:12 No eventing-con process running
2021/12/25 03:27:14 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:27:21 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:27:56 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:27:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/25 03:27:57 Starting bucket ops to source bucket
2021/12/25 03:28:05 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:28:35 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:28:40 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/25 03:28:40 App: TestDeployUndeployLoopTimer got deployed
2021/12/25 03:28:40 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:28:40 Undeploying app: bucket_op_with_timer
2021/12/25 03:28:45 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:28:50 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:28:55 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:29:00 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:29:00 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:29:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:29:00 No eventing-con process running
2021/12/25 03:29:02 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:29:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:29:44 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:29:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/25 03:29:46 Starting bucket ops to source bucket
2021/12/25 03:29:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:29:58 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:08 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:30:23 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:30:28 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/25 03:30:28 App: TestDeployUndeployLoopTimer got deployed
2021/12/25 03:30:28 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:30:29 Undeploying app: bucket_op_with_timer
2021/12/25 03:30:34 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:30:39 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:30:44 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:30:49 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:30:49 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:30:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:30:49 No eventing-con process running
2021/12/25 03:30:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:30:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:31:33 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:31:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/25 03:31:34 Starting bucket ops to source bucket
2021/12/25 03:31:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:31:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:31:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:31:57 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:32:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:32:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:32:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:32:17 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/25 03:32:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:32:22 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/25 03:32:22 App: TestDeployUndeployLoopTimer got deployed
2021/12/25 03:32:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:32:22 Undeploying app: bucket_op_with_timer
2021/12/25 03:32:27 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:32:32 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:32:37 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:32:42 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/25 03:32:42 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/25 03:32:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:32:42 No eventing-con process running
2021/12/25 03:32:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:32:51 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:33:26 Function: TestDeployUndeployLoopTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer deleting in background"
}
--- PASS: TestDeployUndeployLoopTimer (580.60s)
=== RUN   TestMultipleHandlers
2021/12/25 03:33:36 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/25 03:33:36 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/25 03:33:36 No eventing-con process running
2021/12/25 03:33:36 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_update not found"
}
2021/12/25 03:33:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:33:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:33:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:33:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:33:55 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/25 03:33:55 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/25 03:33:55 No eventing-con process running
2021/12/25 03:33:55 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_delete not found"
}
2021/12/25 03:33:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:34:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:34:03 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:34:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:34:08 Deploying app: TestMultipleHandlers_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:34:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update stored in temp store"
}
2021/12/25 03:34:09 Deploying app: TestMultipleHandlers_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:34:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete stored in temp store"
}
2021/12/25 03:34:16 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/25 03:34:21 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/25 03:34:26 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/25 03:34:31 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/25 03:34:31 App: TestMultipleHandlers_bucket_op_on_update got deployed
2021/12/25 03:34:36 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/25 03:34:41 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/25 03:34:46 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/25 03:34:46 App: TestMultipleHandlers_bucket_op_on_delete got deployed
2021/12/25 03:34:46 Starting bucket ops to source bucket
2021/12/25 03:34:56 Waiting for dst bucket item count to get to: 5000 curr count: 642
2021/12/25 03:35:01 Waiting for dst bucket item count to get to: 5000 curr count: 2378
2021/12/25 03:35:06 Waiting for dst bucket item count to get to: 5000 curr count: 4236
2021/12/25 03:35:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:35:06 Starting bucket ops to source bucket
2021/12/25 03:35:21 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:35:21 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 03:35:21 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:35:21 Function: TestMultipleHandlers_bucket_op_on_update update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:35:21 Function: TestMultipleHandlers_bucket_op_on_delete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:35:26 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/25 03:35:31 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/25 03:35:36 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/25 03:35:41 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/25 03:35:41 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/25 03:35:41 No eventing-con process running
2021/12/25 03:35:41 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update deleting in background"
}
2021/12/25 03:35:43 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:35:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:35:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:36:00 Waiting for dst bucket item count to get to: 0 curr count: 5971
2021/12/25 03:36:05 Waiting for dst bucket item count to get to: 0 curr count: 5971
2021/12/25 03:36:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:36:10 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/25 03:36:15 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/25 03:36:20 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/25 03:36:25 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/25 03:36:25 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/25 03:36:25 No eventing-con process running
2021/12/25 03:36:25 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete deleting in background"
}
2021/12/25 03:36:27 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:36:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:36:34 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:36:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMultipleHandlers (192.66s)
=== RUN   TestPauseResumeLoopDefaultSettings
2021/12/25 03:36:49 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/25 03:36:49 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/25 03:36:49 No eventing-con process running
2021/12/25 03:36:49 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopDefaultSettings not found"
}
2021/12/25 03:36:51 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:36:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:36:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:37:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:37:02 Deploying app: TestPauseResumeLoopDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:37:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings stored in temp store"
}
2021/12/25 03:37:04 Starting bucket ops to source bucket
2021/12/25 03:37:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:37:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:37:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:37:27 Waiting for dst bucket item count to get to: 5000 curr count: 2393
2021/12/25 03:37:32 Waiting for dst bucket item count to get to: 5000 curr count: 4893
2021/12/25 03:37:32 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:37:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:37:37 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/25 03:37:37 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/25 03:37:37 Pausing app: bucket_op_on_update
2021/12/25 03:37:37 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:37:42 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/25 03:37:42 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/25 03:37:42 Resuming app: bucket_op_on_update
2021/12/25 03:37:43 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:37:43 Starting bucket ops to source bucket
2021/12/25 03:37:51 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:37:56 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:38:01 Waiting for dst bucket item count to get to: 10000 curr count: 5249
2021/12/25 03:38:06 Waiting for dst bucket item count to get to: 10000 curr count: 7749
2021/12/25 03:38:06 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 03:38:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:38:11 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/25 03:38:11 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/25 03:38:11 Pausing app: bucket_op_on_update
2021/12/25 03:38:11 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:38:16 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/25 03:38:16 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/25 03:38:16 Resuming app: bucket_op_on_update
2021/12/25 03:38:17 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:38:17 Starting bucket ops to source bucket
2021/12/25 03:38:25 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:38:30 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:38:35 Waiting for dst bucket item count to get to: 15000 curr count: 12500
2021/12/25 03:38:35 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/25 03:38:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:38:40 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/25 03:38:40 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/25 03:38:40 Pausing app: bucket_op_on_update
2021/12/25 03:38:40 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:38:45 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/25 03:38:45 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/25 03:38:45 Resuming app: bucket_op_on_update
2021/12/25 03:38:46 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:38:46 Starting bucket ops to source bucket
2021/12/25 03:38:54 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:38:59 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:39:04 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:39:09 Waiting for dst bucket item count to get to: 20000 curr count: 17500
2021/12/25 03:39:09 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/25 03:39:09 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:39:14 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/25 03:39:14 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/25 03:39:14 Pausing app: bucket_op_on_update
2021/12/25 03:39:15 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:39:20 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/25 03:39:20 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/25 03:39:20 Resuming app: bucket_op_on_update
2021/12/25 03:39:21 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:39:21 Starting bucket ops to source bucket
2021/12/25 03:39:29 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:39:34 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:39:39 Waiting for dst bucket item count to get to: 25000 curr count: 22698
2021/12/25 03:39:44 Waiting for dst bucket item count to get to: 25000 curr count: 22698
2021/12/25 03:39:44 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/25 03:39:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:39:49 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/25 03:39:49 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/25 03:39:49 Pausing app: bucket_op_on_update
2021/12/25 03:39:49 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:39:54 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/25 03:39:54 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/25 03:39:59 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/25 03:40:04 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/25 03:40:09 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/25 03:40:14 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/25 03:40:14 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/25 03:40:14 No eventing-con process running
2021/12/25 03:40:14 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings deleting in background"
}
2021/12/25 03:40:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:40:26 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/25 03:40:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:40:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:40:38 Waiting for dst bucket item count to get to: 0 curr count: 14166
2021/12/25 03:40:43 Waiting for dst bucket item count to get to: 0 curr count: 1666
2021/12/25 03:40:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopDefaultSettings (243.80s)
=== RUN   TestPauseResumeLoopNonDefaultSettings
2021/12/25 03:40:53 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/25 03:40:53 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/25 03:40:53 No eventing-con process running
2021/12/25 03:40:53 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopNonDefaultSettings not found"
}
2021/12/25 03:40:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:41:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:41:01 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:41:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:41:06 Deploying app: TestPauseResumeLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:41:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings stored in temp store"
}
2021/12/25 03:41:07 Starting bucket ops to source bucket
2021/12/25 03:41:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:41:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:41:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:41:50 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/25 03:41:50 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/25 03:41:50 Pausing app: bucket_op_with_timer
2021/12/25 03:41:51 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:41:56 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/25 03:41:56 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/25 03:41:56 Resuming app: bucket_op_with_timer
2021/12/25 03:41:57 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:41:57 Starting bucket ops to source bucket
2021/12/25 03:42:05 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:42:10 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:42:15 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:42:20 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:42:25 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:42:30 Waiting for dst bucket item count to get to: 10000 curr count: 7036
2021/12/25 03:42:35 Waiting for dst bucket item count to get to: 10000 curr count: 9536
2021/12/25 03:42:35 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 03:42:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:42:40 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/25 03:42:40 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/25 03:42:40 Pausing app: bucket_op_with_timer
2021/12/25 03:42:40 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:42:45 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/25 03:42:45 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/25 03:42:45 Resuming app: bucket_op_with_timer
2021/12/25 03:42:46 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:42:46 Starting bucket ops to source bucket
2021/12/25 03:42:54 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:42:59 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:43:04 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:43:09 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/25 03:43:14 Waiting for dst bucket item count to get to: 15000 curr count: 10002
2021/12/25 03:43:19 Waiting for dst bucket item count to get to: 15000 curr count: 12502
2021/12/25 03:43:19 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/25 03:43:19 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:43:24 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/25 03:43:24 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/25 03:43:24 Pausing app: bucket_op_with_timer
2021/12/25 03:43:24 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:43:29 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/25 03:43:29 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/25 03:43:29 Resuming app: bucket_op_with_timer
2021/12/25 03:43:30 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:43:30 Starting bucket ops to source bucket
2021/12/25 03:43:38 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:43:43 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:43:48 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:43:53 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/25 03:43:58 Waiting for dst bucket item count to get to: 20000 curr count: 16087
2021/12/25 03:44:03 Waiting for dst bucket item count to get to: 20000 curr count: 17846
2021/12/25 03:44:08 Waiting for dst bucket item count to get to: 20000 curr count: 19259
2021/12/25 03:44:08 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/25 03:44:08 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:44:13 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/25 03:44:13 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/25 03:44:13 Pausing app: bucket_op_with_timer
2021/12/25 03:44:14 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:44:19 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/25 03:44:19 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/25 03:44:19 Resuming app: bucket_op_with_timer
2021/12/25 03:44:20 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:44:20 Starting bucket ops to source bucket
2021/12/25 03:44:28 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:44:33 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:44:38 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:44:43 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/25 03:44:48 Waiting for dst bucket item count to get to: 25000 curr count: 22208
2021/12/25 03:44:53 Waiting for dst bucket item count to get to: 25000 curr count: 22208
2021/12/25 03:44:53 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/25 03:44:53 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:44:58 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/25 03:44:58 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/25 03:44:58 Pausing app: bucket_op_with_timer
2021/12/25 03:44:58 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:45:03 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/25 03:45:03 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/25 03:45:08 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/25 03:45:13 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/25 03:45:18 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/25 03:45:23 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/25 03:45:23 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/25 03:45:23 No eventing-con process running
2021/12/25 03:45:23 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings deleting in background"
}
2021/12/25 03:45:25 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:45:35 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/25 03:45:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:45:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:45:47 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/25 03:45:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopNonDefaultSettings (303.92s)
=== RUN   TestPauseAndResumeWithWorkerCountChange
2021/12/25 03:45:57 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/25 03:45:57 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/25 03:45:57 No eventing-con process running
2021/12/25 03:45:57 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseAndResumeWithWorkerCountChange not found"
}
2021/12/25 03:45:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:46:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:46:05 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:46:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:46:10 Deploying app: TestPauseAndResumeWithWorkerCountChange. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:46:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange stored in temp store"
}
2021/12/25 03:46:16 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/25 03:46:21 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/25 03:46:26 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/25 03:46:31 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/25 03:46:31 App: TestPauseAndResumeWithWorkerCountChange got deployed
2021/12/25 03:46:31 Starting bucket ops to source bucket
2021/12/25 03:46:41 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:46:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:46:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:46:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:47:01 Waiting for dst bucket item count to get to: 5000 curr count: 2317
2021/12/25 03:47:06 Waiting for dst bucket item count to get to: 5000 curr count: 2317
2021/12/25 03:47:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:47:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:47:06 Pausing app: bucket_op_with_timer
2021/12/25 03:47:06 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:47:11 Waiting for function: TestPauseAndResumeWithWorkerCountChange status to change to paused
2021/12/25 03:47:11 Function: TestPauseAndResumeWithWorkerCountChange status changed to paused
2021/12/25 03:47:11 Resuming app: bucket_op_with_timer
2021/12/25 03:47:12 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:47:12 Starting bucket ops to source bucket
2021/12/25 03:47:24 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:47:29 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:47:34 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:47:39 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:47:44 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:47:49 Waiting for dst bucket item count to get to: 10000 curr count: 9320
2021/12/25 03:47:54 Waiting for dst bucket item count to get to: 10000 curr count: 9320
2021/12/25 03:47:54 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 03:47:59 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/25 03:48:04 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/25 03:48:09 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/25 03:48:14 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/25 03:48:14 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/25 03:48:14 No eventing-con process running
2021/12/25 03:48:14 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange deleting in background"
}
2021/12/25 03:48:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:48:26 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 03:48:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:48:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:48:38 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:48:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseAndResumeWithWorkerCountChange (170.88s)
=== RUN   TestPauseResumeWithEventingReb
2021/12/25 03:48:48 Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/25 03:48:48 App: TestPauseResumeWithEventingReb got un-deployed
2021/12/25 03:48:48 No eventing-con process running
2021/12/25 03:48:48 Function: TestPauseResumeWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeWithEventingReb not found"
}
2021/12/25 03:48:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:48:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:48:56 Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:49:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:49:01 Deploying app: TestPauseResumeWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:49:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb stored in temp store"
}
2021/12/25 03:49:07 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/25 03:49:12 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/25 03:49:17 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/25 03:49:22 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/25 03:49:22 App: TestPauseResumeWithEventingReb got deployed
2021/12/25 03:49:22 Starting bucket ops to source bucket
2021/12/25 03:49:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:49:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:49:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:49:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 03:49:52 Waiting for dst bucket item count to get to: 5000 curr count: 4172
2021/12/25 03:49:57 Waiting for dst bucket item count to get to: 5000 curr count: 4172
2021/12/25 03:49:57 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:49:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:49:57 Pausing app: bucket_op_with_timer
2021/12/25 03:49:57 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:50:02 Waiting for function: TestPauseResumeWithEventingReb status to change to paused
2021/12/25 03:50:02 Function: TestPauseResumeWithEventingReb status changed to paused
2021/12/25 03:50:02 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 03:50:02 Removing node(s): [] from the cluster
2021/12/25 03:50:07 Rebalance progress: 29.16666666666666
2021/12/25 03:50:12 Rebalance progress: 29.16666666666666
2021/12/25 03:50:17 Rebalance progress: 100
2021/12/25 03:50:17 Resuming app: bucket_op_with_timer
2021/12/25 03:50:18 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:50:18 Starting bucket ops to source bucket
2021/12/25 03:50:30 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:50:35 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:50:40 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:50:45 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:50:50 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/25 03:50:55 Waiting for dst bucket item count to get to: 10000 curr count: 7500
2021/12/25 03:50:55 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 03:50:55 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 03:51:00 Rebalance progress: 29.16666666666666
2021/12/25 03:51:05 Rebalance progress: 29.16666666666666
2021/12/25 03:51:10 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/25 03:51:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:51:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:51:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:52:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:52:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:52:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:52:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:53:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:53:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:53:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 03:53:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:76} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: b6c55ce1fd5a59d4ceb255346dec6ee7
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [171-227]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [228-284]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 57
	v: [285-341]
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [0-56]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [57-113]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [114-170]
Producer node: 172.31.2.134:9300	Node UUID: fcf7f0c6d9dacb0af9eb312009eb4dd6
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [683-739]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [740-796]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [797-853]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [854-910]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [911-967]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [968-1023]
Producer node: 127.0.0.1:9303	Node UUID: 28208cb83453792b94127141277b33ab
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [342-398]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [399-455]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [456-512]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [513-569]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [570-626]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [627-682]


2021/12/25 03:54:00 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:54:05 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/25 03:54:10 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/25 03:54:15 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/25 03:54:20 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/25 03:54:20 clusterSetup.go:405: App: TestPauseResumeWithEventingReb got un-deployed
2021/12/25 03:54:20 eventingSetup.go:46: No eventing-con process running
2021/12/25 03:54:21 eventingSetup.go:418: Function: TestPauseResumeWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb deleting in background"
}
2021/12/25 03:54:22 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:54:32 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 03:54:37 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:54:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:54:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:54:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 03:54:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:54:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeWithEventingReb (376.62s)
=== RUN   TestBucketDeleteAfterPause
2021/12/25 03:55:04 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/25 03:55:04 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/25 03:55:04 eventingSetup.go:46: No eventing-con process running
2021/12/25 03:55:04 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteAfterPause not found"
}
2021/12/25 03:55:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:55:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:55:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:55:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:55:18 eventingSetup.go:125: Deploying app: TestBucketDeleteAfterPause. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:55:19 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause stored in temp store"
}
2021/12/25 03:55:24 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/25 03:55:29 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/25 03:55:34 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/25 03:55:39 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/25 03:55:39 clusterSetup.go:354: App: TestBucketDeleteAfterPause got deployed
2021/12/25 03:55:39 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 03:55:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 151
2021/12/25 03:55:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2651
2021/12/25 03:55:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2651
2021/12/25 03:55:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 03:55:59 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 03:55:59 handler_test.go:712: Pausing app: bucket_op_on_update
2021/12/25 03:55:59 eventingSetup.go:270: Function: TestBucketDeleteAfterPause update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:56:04 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to paused
2021/12/25 03:56:04 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to paused
2021/12/25 03:56:04 handler_test.go:716: Deleting source bucket: default
2021/12/25 03:56:10 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to undeployed
2021/12/25 03:56:10 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to undeployed
2021/12/25 03:56:15 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/25 03:56:15 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/25 03:56:15 eventingSetup.go:46: No eventing-con process running
2021/12/25 03:56:15 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause deleting in background"
}
2021/12/25 03:56:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:56:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 03:56:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:56:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:56:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4103
2021/12/25 03:56:43 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1603
2021/12/25 03:56:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketDeleteAfterPause (109.16s)
=== RUN   TestChangeFnCodeBetweenPauseResume
2021/12/25 03:56:53 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/25 03:56:53 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/25 03:56:53 eventingSetup.go:46: No eventing-con process running
2021/12/25 03:56:53 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestChangeFnCodeBetweenPauseResume not found"
}
2021/12/25 03:56:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:57:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:57:01 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 03:57:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 03:57:06 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 0 dst bucket: 0
2021/12/25 03:57:08 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/25 03:57:13 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/25 03:57:18 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/25 03:57:23 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/25 03:57:28 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/25 03:57:28 clusterSetup.go:354: App: TestChangeFnCodeBetweenPauseResume got deployed
2021/12/25 03:57:28 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 03:58:03 handler_test.go:739: Pausing function: TestChangeFnCodeBetweenPauseResume
2021/12/25 03:58:03 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 03:58:08 eventingSetup.go:719: Waiting for function: TestChangeFnCodeBetweenPauseResume status to change to paused
2021/12/25 03:58:08 eventingSetup.go:727: Function: TestChangeFnCodeBetweenPauseResume status changed to paused
2021/12/25 04:01:08 handler_test.go:748: Resuming function: TestChangeFnCodeBetweenPauseResume
2021/12/25 04:01:08 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 5000 dst bucket: 0
2021/12/25 04:01:09 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/25 04:01:14 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/25 04:01:14 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:01:14 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 0 expected 5000
2021/12/25 04:01:19 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/25 04:01:19 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:01:19 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 2752 expected 5000
2021/12/25 04:01:24 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/25 04:01:24 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:01:24 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 3353 expected 5000
2021/12/25 04:01:29 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/25 04:01:29 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:01:29 eventingSetup.go:804: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 5000
2021/12/25 04:01:29 handler_test.go:753: Undeploying function: TestChangeFnCodeBetweenPauseResume
2021/12/25 04:01:29 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:01:39 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/25 04:01:44 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/25 04:01:49 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/25 04:01:49 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/25 04:01:49 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:01:49 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume deleting in background"
}
2021/12/25 04:01:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:02:01 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:02:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:02:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:02:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:02:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestChangeFnCodeBetweenPauseResume (329.03s)
=== RUN   TestCommentUnCommentOnDelete
2021/12/25 04:02:22 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/25 04:02:22 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/25 04:02:22 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:02:22 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCommentUnCommentOnDelete not found"
}
2021/12/25 04:02:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:02:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:02:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:02:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:02:35 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:02:37 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/25 04:02:42 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:02:47 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:02:52 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:02:57 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:02:57 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/25 04:02:57 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:03:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:03:11 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 1789
2021/12/25 04:03:16 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 1789
2021/12/25 04:03:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:03:16 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:03:16 handler_test.go:780: Undeploying app: TestCommentUnCommentOnDelete
2021/12/25 04:03:16 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:03:46 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 5000 dst bucket: 5000
2021/12/25 04:03:47 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/25 04:03:52 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:03:57 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:04:02 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:04:07 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/25 04:04:07 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/25 04:04:07 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:04:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:04:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:04:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:04:25 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:04:25 handler_test.go:799: Undeploying app: TestCommentUnCommentOnDelete
2021/12/25 04:04:25 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:04:35 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/25 04:04:40 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/25 04:04:45 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/25 04:04:45 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/25 04:04:45 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:04:45 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete deleting in background"
}
2021/12/25 04:04:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:04:51 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:04:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:04:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCommentUnCommentOnDelete (165.77s)
=== RUN   TestCPPWorkerCleanup
2021/12/25 04:05:08 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/25 04:05:08 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/25 04:05:08 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:05:08 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCPPWorkerCleanup not found"
}
2021/12/25 04:05:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:05:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:05:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:05:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:05:21 eventingSetup.go:125: Deploying app: TestCPPWorkerCleanup. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:05:22 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup stored in temp store"
}
2021/12/25 04:05:22 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:05:29 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:05:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:05:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:05:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:05:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 3013
2021/12/25 04:05:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 3013
2021/12/25 04:05:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:05:54 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:06:00 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/25 04:06:05 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/25 04:06:10 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/25 04:06:15 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/25 04:06:15 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/25 04:06:15 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:06:15 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup deleting in background"
}
2021/12/25 04:06:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:06:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:06:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:06:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:06:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:06:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCPPWorkerCleanup (129.52s)
=== RUN   TestWithUserXattrs
2021/12/25 04:07:18 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/25 04:07:18 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/25 04:07:18 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:07:18 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestWithUserXattrs not found"
}
2021/12/25 04:07:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:07:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:07:26 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:07:31 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:07:31 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:07:33 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:07:35 eventingSetup.go:125: Deploying app: TestWithUserXattrs. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:07:36 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs stored in temp store"
}
2021/12/25 04:07:41 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/25 04:07:46 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/25 04:07:51 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/25 04:07:56 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/25 04:07:56 clusterSetup.go:354: App: TestWithUserXattrs got deployed
2021/12/25 04:07:56 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:08:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:08:10 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2480
2021/12/25 04:08:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2480
2021/12/25 04:08:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:08:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:08:20 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/25 04:08:26 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/25 04:08:31 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/25 04:08:36 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/25 04:08:36 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/25 04:08:36 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:08:36 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs deleting in background"
}
2021/12/25 04:08:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:08:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:08:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:08:49 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:08:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4615
2021/12/25 04:09:04 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2115
2021/12/25 04:09:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestWithUserXattrs (116.07s)
=== RUN   TestEventProcessingPostBucketFlush
2021/12/25 04:09:14 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:09:14 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/25 04:09:14 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:09:14 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventProcessingPostBucketFlush not found"
}
2021/12/25 04:09:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:09:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:09:22 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:09:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:09:27 eventingSetup.go:125: Deploying app: TestEventProcessingPostBucketFlush. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:09:28 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush stored in temp store"
}
2021/12/25 04:09:33 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/25 04:09:38 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/25 04:09:43 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/25 04:09:48 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/25 04:09:48 clusterSetup.go:354: App: TestEventProcessingPostBucketFlush got deployed
2021/12/25 04:09:48 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:09:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:10:02 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 768
2021/12/25 04:10:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 768
2021/12/25 04:10:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:10:07 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:10:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:10:21 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3098
2021/12/25 04:10:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3098
2021/12/25 04:10:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:10:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:10:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3422
2021/12/25 04:10:43 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 922
2021/12/25 04:10:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:10:43 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:11:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 12912
2021/12/25 04:11:13 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 19219
2021/12/25 04:11:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 22759
2021/12/25 04:11:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/25 04:11:18 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:11:18 eventingSetup.go:270: Function: TestEventProcessingPostBucketFlush update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:11:23 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:11:28 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:11:33 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:11:38 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:11:38 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/25 04:11:38 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:11:43 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/25 04:11:43 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/25 04:11:43 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:11:43 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush deleting in background"
}
2021/12/25 04:11:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:11:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/25 04:11:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:11:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:12:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/25 04:12:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventProcessingPostBucketFlush (182.51s)
=== RUN   TestMetaBucketDelete
2021/12/25 04:12:16 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/25 04:12:16 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/25 04:12:16 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:12:16 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDelete not found"
}
2021/12/25 04:12:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:12:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:12:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:12:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:12:29 eventingSetup.go:125: Deploying app: TestMetaBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:12:30 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete stored in temp store"
}
2021/12/25 04:12:35 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/25 04:12:40 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/25 04:12:45 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/25 04:12:50 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/25 04:12:50 clusterSetup.go:354: App: TestMetaBucketDelete got deployed
2021/12/25 04:12:50 handler_test.go:902: Deleting metadata bucket: eventing
2021/12/25 04:12:51 handler_test.go:904: Deleted metadata bucket: eventing
2021/12/25 04:12:56 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/25 04:12:56 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/25 04:13:12 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/25 04:13:12 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/25 04:13:12 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:13:12 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete deleting in background"
}
2021/12/25 04:13:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:13:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:13:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:13:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDelete (78.38s)
=== RUN   TestMetaBucketDeleteWithBootstrap
2021/12/25 04:13:35 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:13:35 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:13:35 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:13:35 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDeleteWithBootstrap not found"
}
2021/12/25 04:13:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:13:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:13:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:13:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:13:48 eventingSetup.go:125: Deploying app: TestMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:13:49 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/25 04:14:08 handler_test.go:923: Deleting metadata bucket: eventing
2021/12/25 04:14:08 handler_test.go:925: Deleted metadata bucket: eventing
2021/12/25 04:14:23 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:14:23 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:14:38 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:14:38 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:14:38 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:14:38 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/25 04:14:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:14:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:14:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:14:51 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDeleteWithBootstrap (86.79s)
=== RUN   TestSourceBucketDelete
2021/12/25 04:15:01 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/25 04:15:01 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/25 04:15:01 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:15:01 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDelete not found"
}
2021/12/25 04:15:03 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:15:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:15:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:15:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:15:14 eventingSetup.go:125: Deploying app: TestSourceBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:15:16 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete stored in temp store"
}
2021/12/25 04:15:21 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/25 04:15:26 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/25 04:15:31 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/25 04:15:36 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/25 04:15:36 clusterSetup.go:354: App: TestSourceBucketDelete got deployed
2021/12/25 04:15:36 handler_test.go:945: Deleting source bucket: default
2021/12/25 04:15:37 handler_test.go:947: Deleted source bucket: default
2021/12/25 04:15:42 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/25 04:15:43 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/25 04:15:58 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/25 04:15:58 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/25 04:15:58 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:15:58 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete deleting in background"
}
2021/12/25 04:15:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:16:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:16:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:16:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDelete (79.29s)
=== RUN   TestSourceBucketDeleteWithBootstrap
2021/12/25 04:16:21 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:16:21 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:16:21 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:16:21 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDeleteWithBootstrap not found"
}
2021/12/25 04:16:22 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:16:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:16:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:16:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:16:34 eventingSetup.go:125: Deploying app: TestSourceBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:16:35 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap stored in temp store"
}
2021/12/25 04:16:54 handler_test.go:966: Deleting source bucket: default
2021/12/25 04:16:55 handler_test.go:968: Deleted source bucket: default
2021/12/25 04:17:10 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:17:10 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:17:25 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:17:25 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:17:25 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:17:25 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap deleting in background"
}
2021/12/25 04:17:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:17:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:17:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:17:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDeleteWithBootstrap (87.67s)
=== RUN   TestSourceAndMetaBucketDeleteWithBootstrap
2021/12/25 04:17:48 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:17:48 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:17:48 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:17:48 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceAndMetaBucketDeleteWithBootstrap not found"
}
2021/12/25 04:17:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:17:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:17:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:18:01 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:18:01 eventingSetup.go:125: Deploying app: TestSourceAndMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:18:03 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/25 04:18:21 handler_test.go:988: Deleting source bucket: default
2021/12/25 04:18:23 handler_test.go:990: Deleted source bucket: default
2021/12/25 04:18:23 handler_test.go:992: Deleting metadata bucket: eventing
2021/12/25 04:18:23 handler_test.go:994: Deleted metadata bucket: eventing
2021/12/25 04:18:38 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:18:38 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:18:54 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/25 04:18:54 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/25 04:18:54 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:18:54 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/25 04:18:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:19:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:19:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:19:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceAndMetaBucketDeleteWithBootstrap (88.21s)
=== RUN   TestUndeployDuringBootstrap
2021/12/25 04:19:17 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/25 04:19:17 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/25 04:19:17 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:19:17 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployDuringBootstrap not found"
}
2021/12/25 04:19:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:19:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:19:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:19:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:19:30 eventingSetup.go:125: Deploying app: TestUndeployDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:19:31 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap stored in temp store"
}
2021/12/25 04:19:31 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:19:43 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:19:43 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/25 04:19:43 clusterSetup.go:378: App: TestUndeployDuringBootstrap started bootstrap
2021/12/25 04:19:43 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/25 04:19:48 clusterSetup.go:367: No apps undergoing bootstrap
2021/12/25 04:19:53 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/25 04:19:58 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/25 04:20:03 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/25 04:20:03 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/25 04:20:03 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:20:28 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/25 04:20:28 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/25 04:20:28 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:20:28 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap deleting in background"
}
2021/12/25 04:20:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:20:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:20:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:20:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:20:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:20:56 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2126
2021/12/25 04:20:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployDuringBootstrap (109.54s)
=== RUN   TestDeleteBeforeUndeploy
2021/12/25 04:21:01 eventingSetup.go:125: Deploying app: TestDeleteBeforeUndeploy. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:21:02 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy stored in temp store"
}
2021/12/25 04:21:07 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/25 04:21:12 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/25 04:21:17 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/25 04:21:22 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/25 04:21:22 clusterSetup.go:354: App: TestDeleteBeforeUndeploy got deployed
2021/12/25 04:21:22 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 422 dump: {
 "name": "ERR_APP_NOT_UNDEPLOYED",
 "code": 22,
 "description": "Function: TestDeleteBeforeUndeploy skipping delete request as it hasn't been undeployed"
}
2021/12/25 04:21:22 eventingSetup.go:270: Function: TestDeleteBeforeUndeploy update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:21:23 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy deleting in background"
}
2021/12/25 04:21:28 clusterSetup.go:397: Waiting for app: TestDeleteBeforeUndeploy to get un-deployed
2021/12/25 04:21:28 clusterSetup.go:405: App: TestDeleteBeforeUndeploy got un-deployed
2021/12/25 04:21:28 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:21:28 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeleteBeforeUndeploy not found"
}
2021/12/25 04:21:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:21:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:21:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:21:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDeleteBeforeUndeploy (44.68s)
=== RUN   TestUndeployWhenTimersAreFired
2021/12/25 04:21:51 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:21:51 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/25 04:21:51 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:21:51 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWhenTimersAreFired not found"
}
2021/12/25 04:21:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:21:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:21:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:22:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:22:04 eventingSetup.go:125: Deploying app: TestUndeployWhenTimersAreFired. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:22:05 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired stored in temp store"
}
2021/12/25 04:22:10 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/25 04:22:15 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/25 04:22:20 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/25 04:22:25 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/25 04:22:25 clusterSetup.go:354: App: TestUndeployWhenTimersAreFired got deployed
2021/12/25 04:22:25 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:23:01 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:23:06 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:23:11 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:23:16 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:23:16 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/25 04:23:16 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:25:01 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/25 04:25:01 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/25 04:25:01 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:25:01 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired deleting in background"
}
2021/12/25 04:25:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:25:12 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:25:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:25:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:25:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:25:29 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:25:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:25:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWhenTimersAreFired (233.23s)
=== RUN   TestUndeployWithKVFailover
2021/12/25 04:25:44 clusterSetup.go:397: Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/25 04:25:44 clusterSetup.go:405: App: TestUndeployWithKVFailover got un-deployed
2021/12/25 04:25:44 eventingSetup.go:46: No eventing-con process running
2021/12/25 04:25:44 eventingSetup.go:418: Function: TestUndeployWithKVFailover delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWithKVFailover not found"
}
2021/12/25 04:25:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:25:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:25:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:25:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:25:57 eventingSetup.go:125: Deploying app: TestUndeployWithKVFailover. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:25:58 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover stored in temp store"
}
2021/12/25 04:25:58 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 04:26:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:10 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:30 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:26:35 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2469
2021/12/25 04:26:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4969
2021/12/25 04:26:40 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:26:40 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/25 04:26:41 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 04:26:46 Rebalance progress: 29.16666666666666
2021/12/25 04:26:51 Rebalance progress: 29.16666666666666
2021/12/25 04:26:56 Rebalance progress: 100
2021/12/25 04:26:56 Failing over: [127.0.0.1:19003]
2021/12/25 04:27:06 Function: TestUndeployWithKVFailover update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/25 04:28:06 Removing node(s): [] from the cluster
2021/12/25 04:28:11 Rebalance progress: 29.16666666666666
2021/12/25 04:28:16 Rebalance progress: 29.16666666666666
2021/12/25 04:28:21 Rebalance progress: 100
2021/12/25 04:28:26 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/25 04:28:26 App: TestUndeployWithKVFailover got un-deployed
2021/12/25 04:28:26 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:28:31 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/25 04:28:31 App: TestUndeployWithKVFailover got un-deployed
2021/12/25 04:28:31 No eventing-con process running
2021/12/25 04:28:31 Function: TestUndeployWithKVFailover delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover deleting in background"
}
2021/12/25 04:28:32 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:28:42 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:28:47 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:28:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:28:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:28:59 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:29:04 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:29:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWithKVFailover (209.92s)
=== RUN   TestBucketFlushWhileFnDeployed
2021/12/25 04:29:14 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/25 04:29:14 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/25 04:29:14 No eventing-con process running
2021/12/25 04:29:14 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketFlushWhileFnDeployed not found"
}
2021/12/25 04:29:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:29:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:29:22 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:29:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:29:27 Deploying app: TestBucketFlushWhileFnDeployed. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:29:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed stored in temp store"
}
2021/12/25 04:29:33 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/25 04:29:38 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/25 04:29:43 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/25 04:29:48 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/25 04:29:48 App: TestBucketFlushWhileFnDeployed got deployed
2021/12/25 04:29:48 Starting bucket ops to source bucket
2021/12/25 04:30:09 Waiting for dst bucket item count to get to: 20000 curr count: 12868
2021/12/25 04:30:14 Waiting for dst bucket item count to get to: 20000 curr count: 17114
2021/12/25 04:30:14 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/25 04:30:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:30:17 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:30:27 Waiting for dst bucket item count to get to: 0 curr count: 20000
2021/12/25 04:30:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:30:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:30:39 Waiting for dst bucket item count to get to: 0 curr count: 17698
2021/12/25 04:30:44 Waiting for dst bucket item count to get to: 0 curr count: 7698
2021/12/25 04:30:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:30:44 Starting bucket ops to source bucket
2021/12/25 04:30:56 Waiting for dst bucket item count to get to: 10000 curr count: 2327
2021/12/25 04:31:01 Waiting for dst bucket item count to get to: 10000 curr count: 6046
2021/12/25 04:31:06 Waiting for dst bucket item count to get to: 10000 curr count: 8719
2021/12/25 04:31:06 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 04:31:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:31:11 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/25 04:31:16 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/25 04:31:21 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/25 04:31:27 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/25 04:31:27 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/25 04:31:27 No eventing-con process running
2021/12/25 04:31:27 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed deleting in background"
}
2021/12/25 04:31:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:31:38 Waiting for dst bucket item count to get to: 0 curr count: 5983
2021/12/25 04:31:43 Waiting for dst bucket item count to get to: 0 curr count: 983
2021/12/25 04:31:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:31:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:31:55 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 04:31:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketFlushWhileFnDeployed (170.62s)
=== RUN   TestTimerOverwrite
2021/12/25 04:32:05 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/25 04:32:05 App: TestTimerOverwrite got un-deployed
2021/12/25 04:32:05 No eventing-con process running
2021/12/25 04:32:05 Function: TestTimerOverwrite delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverwrite not found"
}
2021/12/25 04:32:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:32:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:32:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:32:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:32:18 Deploying app: TestTimerOverwrite. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:32:19 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite stored in temp store"
}
2021/12/25 04:32:24 Waiting for app: TestTimerOverwrite to get deployed
2021/12/25 04:32:29 Waiting for app: TestTimerOverwrite to get deployed
2021/12/25 04:32:34 Waiting for app: TestTimerOverwrite to get deployed
2021/12/25 04:32:39 Waiting for app: TestTimerOverwrite to get deployed
2021/12/25 04:32:39 App: TestTimerOverwrite got deployed
2021/12/25 04:32:49 Starting bucket ops to source bucket
2021/12/25 04:32:58 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:08 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:28 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:38 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 04:33:48 Waiting for dst bucket item count to get to: 5000 curr count: 312
2021/12/25 04:33:53 Waiting for dst bucket item count to get to: 5000 curr count: 627
2021/12/25 04:33:58 Waiting for dst bucket item count to get to: 5000 curr count: 2849
2021/12/25 04:33:58 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:34:08 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:34:13 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/25 04:34:18 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/25 04:34:23 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/25 04:34:28 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/25 04:34:28 App: TestTimerOverwrite got un-deployed
2021/12/25 04:34:28 No eventing-con process running
2021/12/25 04:34:28 Function: TestTimerOverwrite delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite deleting in background"
}
2021/12/25 04:34:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:34:40 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:34:45 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:34:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:34:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:34:56 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:34:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverwrite (181.86s)
=== RUN   TestUndeployBackdoorDuringBootstrap
2021/12/25 04:35:01 Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/25 04:35:02 Removing node(s): [] from the cluster
2021/12/25 04:35:07 Rebalance progress: 100
2021/12/25 04:35:07 Failing over: [127.0.0.1:19003]
2021/12/25 04:35:12 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/25 04:35:12 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/25 04:35:12 No eventing-con process running
2021/12/25 04:35:12 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/25 04:35:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:35:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:35:20 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:35:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:35:25 Deploying app: TestUndeployBackdoorDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:35:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap stored in temp store"
}
2021/12/25 04:35:26 Starting bucket ops to source bucket
2021/12/25 04:35:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:35:36 Context: Post to set retry counter, response code: 200 dump: {
 "code": 0
}
2021/12/25 04:35:36 set retry counter, response &{[123 10 32 34 99 111 100 101 34 58 32 48 10 125] }
2021/12/25 04:36:41 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/25 04:36:41 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/25 04:36:41 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:36:41 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap deleting in background"
}
2021/12/25 04:36:46 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/25 04:36:46 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/25 04:36:46 No eventing-con process running
2021/12/25 04:36:46 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/25 04:36:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:36:57 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:36:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:36:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:37:09 Waiting for dst bucket item count to get to: 0 curr count: 2731
2021/12/25 04:37:14 Waiting for dst bucket item count to get to: 0 curr count: 1060
2021/12/25 04:37:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:37:14 Removing node(s): [] from the cluster
2021/12/25 04:37:19 Rebalance progress: 100
--- PASS: TestUndeployBackdoorDuringBootstrap (142.90s)
=== RUN   TestOnUpdateSrcMutation
2021/12/25 04:37:29 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/25 04:37:29 App: TestOnUpdateSrcMutation got un-deployed
2021/12/25 04:37:29 No eventing-con process running
2021/12/25 04:37:29 Function: TestOnUpdateSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutation not found"
}
2021/12/25 04:37:31 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:37:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:37:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:37:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:37:42 Deploying app: TestOnUpdateSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:37:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation stored in temp store"
}
2021/12/25 04:37:44 Starting bucket ops to source bucket
2021/12/25 04:38:11 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 04:38:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:38:16 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/25 04:38:21 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/25 04:38:26 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/25 04:38:31 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/25 04:38:31 App: TestOnUpdateSrcMutation got un-deployed
2021/12/25 04:38:31 No eventing-con process running
2021/12/25 04:38:31 Function: TestOnUpdateSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation deleting in background"
}
2021/12/25 04:38:32 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:38:42 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 04:38:47 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 04:38:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:38:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:38:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutation (94.52s)
=== RUN   TestOnDeleteSrcMutation
2021/12/25 04:39:04 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/25 04:39:04 App: TestOnDeleteSrcMutation got un-deployed
2021/12/25 04:39:04 No eventing-con process running
2021/12/25 04:39:04 Function: TestOnDeleteSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutation not found"
}
2021/12/25 04:39:05 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:39:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:39:12 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:39:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:39:17 Deploying app: TestOnDeleteSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:39:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation stored in temp store"
}
2021/12/25 04:39:18 Starting bucket ops to source bucket
2021/12/25 04:39:47 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:39:47 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:39:52 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/25 04:39:57 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/25 04:40:02 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/25 04:40:07 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/25 04:40:07 App: TestOnDeleteSrcMutation got un-deployed
2021/12/25 04:40:07 No eventing-con process running
2021/12/25 04:40:07 Function: TestOnDeleteSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation deleting in background"
}
2021/12/25 04:40:08 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:40:18 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:40:23 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:40:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:40:25 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:40:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutation (96.34s)
=== RUN   TestOnUpdateSrcMutationWithTimer
2021/12/25 04:40:40 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/25 04:40:40 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/25 04:40:40 No eventing-con process running
2021/12/25 04:40:40 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutationWithTimer not found"
}
2021/12/25 04:40:42 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:40:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:40:48 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:40:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:40:53 Deploying app: TestOnUpdateSrcMutationWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:40:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer stored in temp store"
}
2021/12/25 04:40:54 Starting bucket ops to source bucket
2021/12/25 04:41:36 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 04:41:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:41:42 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/25 04:41:47 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/25 04:41:52 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/25 04:41:57 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/25 04:41:57 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/25 04:41:57 No eventing-con process running
2021/12/25 04:41:57 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer deleting in background"
}
2021/12/25 04:41:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:42:08 Waiting for dst bucket item count to get to: 0 curr count: 7419
2021/12/25 04:42:13 Waiting for dst bucket item count to get to: 0 curr count: 2419
2021/12/25 04:42:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:42:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:42:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutationWithTimer (109.55s)
=== RUN   TestOnDeleteSrcMutationsWithTimer
2021/12/25 04:42:30 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/25 04:42:30 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/25 04:42:30 No eventing-con process running
2021/12/25 04:42:30 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutationsWithTimer not found"
}
2021/12/25 04:42:31 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:42:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:42:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:42:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:42:43 Deploying app: TestOnDeleteSrcMutationsWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:42:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer stored in temp store"
}
2021/12/25 04:42:44 Starting bucket ops to source bucket
2021/12/25 04:43:28 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 04:43:28 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 04:43:33 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/25 04:43:38 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/25 04:43:43 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/25 04:43:48 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/25 04:43:48 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/25 04:43:48 No eventing-con process running
2021/12/25 04:43:48 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer deleting in background"
}
2021/12/25 04:43:49 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:43:59 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 04:44:04 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 04:44:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:44:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:44:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutationsWithTimer (111.32s)
=== RUN   TestInterHandlerRecursion
2021/12/25 04:44:21 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:44:21 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/25 04:44:21 No eventing-con process running
2021/12/25 04:44:21 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/25 04:44:22 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:44:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:44:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:44:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:44:39 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:44:39 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/25 04:44:39 No eventing-con process running
2021/12/25 04:44:39 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/25 04:44:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:44:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:44:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:44:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:44:52 Deploying app: TestInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:44:53 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/25 04:44:53 response body {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/25 04:44:58 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:45:04 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:45:09 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:45:14 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:45:14 App: TestInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/25 04:45:14 Deploying app: TestInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:45:14 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
}
2021/12/25 04:45:14 response body {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
} err 
2021/12/25 04:45:19 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:45:24 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:45:29 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:45:34 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:45:34 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/25 04:45:34 No eventing-con process running
2021/12/25 04:45:34 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/25 04:45:35 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:45:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:45:42 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:45:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:45:52 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:45:52 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/25 04:45:52 No eventing-con process running
2021/12/25 04:45:52 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/25 04:45:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:45:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:46:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:46:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterHandlerRecursion (113.74s)
=== RUN   TestInterBucketRecursion
2021/12/25 04:46:10 Deploying app: TestInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:46:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
}
2021/12/25 04:46:11 response body {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/25 04:46:11 Deploying app: TestInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:46:11 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion_function2 causes a cycle involving functions: [TestInterBucketRecursion_function1], hence deployment is disallowed"
}
2021/12/25 04:46:16 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/25 04:46:21 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/25 04:46:26 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/25 04:46:31 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/25 04:46:31 App: TestInterBucketRecursion_function1 got deployed
2021/12/25 04:46:36 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:46:41 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:46:46 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:46:51 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:46:51 App: TestInterBucketRecursion_function1 got un-deployed
2021/12/25 04:46:51 No eventing-con process running
2021/12/25 04:46:51 Function: TestInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 deleting in background"
}
2021/12/25 04:46:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:46:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:46:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:47:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:47:09 Waiting for app: TestInterBucketRecursion_function2 to get un-deployed
2021/12/25 04:47:09 App: TestInterBucketRecursion_function2 got un-deployed
2021/12/25 04:47:09 No eventing-con process running
2021/12/25 04:47:09 Function: TestInterBucketRecursion_function2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion_function2 not found"
}
2021/12/25 04:47:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:47:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:47:17 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:47:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion (77.65s)
=== RUN   TestLargeHandler
2021/12/25 04:47:22 Changed force_compress value to false
2021/12/25 04:47:22 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:47:23 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_APPCODE_SIZE",
 "code": 39,
 "description": "Function: TestLargeHandler_function handler Code size is more than 131072. Code Size: 131768"
}
2021/12/25 04:47:23 Changed force_compress value to true
2021/12/25 04:47:23 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:47:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function stored in temp store"
}
2021/12/25 04:47:29 Waiting for app: TestLargeHandler_function to get deployed
2021/12/25 04:47:34 Waiting for app: TestLargeHandler_function to get deployed
2021/12/25 04:47:39 Waiting for app: TestLargeHandler_function to get deployed
2021/12/25 04:47:44 Waiting for app: TestLargeHandler_function to get deployed
2021/12/25 04:47:44 App: TestLargeHandler_function got deployed
2021/12/25 04:47:50 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/25 04:47:55 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/25 04:48:00 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/25 04:48:05 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/25 04:48:05 App: TestLargeHandler_function got un-deployed
2021/12/25 04:48:05 No eventing-con process running
2021/12/25 04:48:05 Function: TestLargeHandler_function delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function deleting in background"
}
2021/12/25 04:48:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:48:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestLargeHandler (55.40s)
=== RUN   TestAllowInterHandlerRecursion
2021/12/25 04:48:28 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:48:28 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/25 04:48:28 No eventing-con process running
2021/12/25 04:48:28 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/25 04:48:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:48:36 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:48:46 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:48:46 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/25 04:48:46 No eventing-con process running
2021/12/25 04:48:46 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/25 04:48:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:48:54 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:48:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:48:59 Deploying app: TestAllowInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:49:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/25 04:49:00 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/25 04:49:05 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:49:10 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:49:15 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:49:20 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/25 04:49:20 App: TestAllowInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/25 04:49:20 Deploying app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:49:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
}
2021/12/25 04:49:22 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
} err 
2021/12/25 04:49:27 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/25 04:49:32 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/25 04:49:37 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/25 04:49:42 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/25 04:49:42 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got deployed
2021/12/25 04:49:42 TestAllowInterHandlerRecursion_src_bucket_op_on_delete is deployed
2021/12/25 04:49:47 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:49:52 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:49:57 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:50:02 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/25 04:50:02 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/25 04:50:02 No eventing-con process running
2021/12/25 04:50:02 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/25 04:50:04 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:50:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:50:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:50:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:50:21 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:50:26 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:50:31 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:50:36 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/25 04:50:36 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/25 04:50:36 No eventing-con process running
2021/12/25 04:50:36 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete deleting in background"
}
2021/12/25 04:50:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:50:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:50:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:50:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterHandlerRecursion (151.49s)
=== RUN   TestAllowInterBucketRecursion
2021/12/25 04:50:54 Deploying app: TestAllowInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:50:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
}
2021/12/25 04:50:56 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/25 04:51:01 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/25 04:51:06 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/25 04:51:11 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/25 04:51:16 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/25 04:51:16 App: TestAllowInterBucketRecursion_function1 got deployed
2021/12/25 04:51:16 Deploying app: TestAllowInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:51:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
}
2021/12/25 04:51:17 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
} err 
2021/12/25 04:51:22 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/25 04:51:27 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/25 04:51:32 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/25 04:51:37 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/25 04:51:37 App: TestAllowInterBucketRecursion_function2 got deployed
2021/12/25 04:51:37 TestAllowInterBucketRecursion_function2 is deployed
2021/12/25 04:51:42 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:51:47 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:51:52 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:51:57 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/25 04:51:57 App: TestAllowInterBucketRecursion_function1 got un-deployed
2021/12/25 04:51:57 No eventing-con process running
2021/12/25 04:51:57 Function: TestAllowInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 deleting in background"
}
2021/12/25 04:51:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:52:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:52:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:52:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:52:17 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/25 04:52:22 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/25 04:52:27 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/25 04:52:32 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/25 04:52:32 App: TestAllowInterBucketRecursion_function2 got un-deployed
2021/12/25 04:52:32 No eventing-con process running
2021/12/25 04:52:32 Function: TestAllowInterBucketRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 deleting in background"
}
2021/12/25 04:52:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:52:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:52:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:52:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterBucketRecursion (115.54s)
=== RUN   TestInterBucketRecursion2
2021/12/25 04:52:51 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:52:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:53:03 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:53:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:53:14 Deploying app: TestInterBucketRecursion2_function1. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:53:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 stored in temp store"
}
2021/12/25 04:53:20 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/25 04:53:25 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/25 04:53:30 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/25 04:53:35 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/25 04:53:35 App: TestInterBucketRecursion2_function1 got deployed
2021/12/25 04:53:35 Deploying app: TestInterBucketRecursion2_function2. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:53:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 stored in temp store"
}
2021/12/25 04:53:42 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/25 04:53:47 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/25 04:53:52 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/25 04:53:57 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/25 04:53:57 App: TestInterBucketRecursion2_function2 got deployed
2021/12/25 04:53:57 Deploying app: TestInterBucketRecursion2_function3. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:53:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 stored in temp store"
}
2021/12/25 04:54:03 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/25 04:54:08 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/25 04:54:13 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/25 04:54:18 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/25 04:54:18 App: TestInterBucketRecursion2_function3 got deployed
2021/12/25 04:54:18 Deploying app: TestInterBucketRecursion2_function4. Item count src bucket: 0 dst bucket: 0
2021/12/25 04:54:18 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion2_function4 causes a cycle involving functions: [TestInterBucketRecursion2_function1 TestInterBucketRecursion2_function2 TestInterBucketRecursion2_function3], hence deployment is disallowed"
}
2021/12/25 04:54:23 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/25 04:54:28 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/25 04:54:33 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/25 04:54:38 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/25 04:54:38 App: TestInterBucketRecursion2_function1 got un-deployed
2021/12/25 04:54:38 No eventing-con process running
2021/12/25 04:54:38 Function: TestInterBucketRecursion2_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 deleting in background"
}
2021/12/25 04:54:39 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:54:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:54:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:54:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:54:56 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/25 04:55:01 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/25 04:55:06 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/25 04:55:11 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/25 04:55:11 App: TestInterBucketRecursion2_function2 got un-deployed
2021/12/25 04:55:11 No eventing-con process running
2021/12/25 04:55:11 Function: TestInterBucketRecursion2_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 deleting in background"
}
2021/12/25 04:55:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:55:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:55:20 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:55:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:55:30 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/25 04:55:35 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/25 04:55:40 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/25 04:55:45 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/25 04:55:45 App: TestInterBucketRecursion2_function3 got un-deployed
2021/12/25 04:55:45 No eventing-con process running
2021/12/25 04:55:45 Function: TestInterBucketRecursion2_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 deleting in background"
}
2021/12/25 04:55:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:55:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:55:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:55:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:56:03 Waiting for app: TestInterBucketRecursion2_function4 to get un-deployed
2021/12/25 04:56:03 App: TestInterBucketRecursion2_function4 got un-deployed
2021/12/25 04:56:03 No eventing-con process running
2021/12/25 04:56:03 Function: TestInterBucketRecursion2_function4 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion2_function4 not found"
}
2021/12/25 04:56:05 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:56:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 04:56:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 04:56:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion2 (213.32s)
=== RUN   TestBucketDeleteWithRebOut
2021/12/25 04:56:23 Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/25 04:56:28 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/25 04:56:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:56:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:57:01 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:57:16 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:57:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:57:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:58:01 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:58:16 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:58:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:58:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:59:01 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:59:16 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:59:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 04:59:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:00:01 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:00:16 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:00:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:00:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:01:01 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:01:16 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:01:31 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/25 05:01:46 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:46} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:02:01 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:02:01 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 05:02:02 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 05:02:07 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:02:07 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:02:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 05:02:07 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 05:02:12 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:02:12 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:02:17 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/25 05:02:17 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/25 05:02:17 eventingSetup.go:46: No eventing-con process running
2021/12/25 05:02:17 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteWithRebOut not found"
}
2021/12/25 05:02:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:02:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:02:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:02:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:02:35 eventingSetup.go:125: Deploying app: TestBucketDeleteWithRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:02:36 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut stored in temp store"
}
2021/12/25 05:02:41 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/25 05:02:46 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/25 05:02:52 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/25 05:02:57 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/25 05:02:57 clusterSetup.go:354: App: TestBucketDeleteWithRebOut got deployed

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: b6c55ce1fd5a59d4ceb255346dec6ee7
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]
Producer node: 172.31.2.134:9300	Node UUID: fcf7f0c6d9dacb0af9eb312009eb4dd6
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
Producer node: 127.0.0.1:9303	Node UUID: 28208cb83453792b94127141277b33ab
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]


2021/12/25 05:02:57 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:02:57 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 05:03:02 Rebalance progress: 29.16666666666666
2021/12/25 05:03:07 Rebalance progress: 29.16666666666666
2021/12/25 05:03:12 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: b6c55ce1fd5a59d4ceb255346dec6ee7
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]
Producer node: 172.31.2.134:9300	Node UUID: fcf7f0c6d9dacb0af9eb312009eb4dd6
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
Producer node: 127.0.0.1:9303	Node UUID: 28208cb83453792b94127141277b33ab
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]


2021/12/25 05:03:12 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:03:12 handler_test.go:1674: Deleting source bucket: default
2021/12/25 05:03:13 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 05:03:14 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 05:03:19 Rebalance progress: 29.16666666666666
2021/12/25 05:03:24 Rebalance progress: 29.16666666666666
2021/12/25 05:03:29 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 895
	vb list dump: [0-3, 5-14, 16-21, 23-25, 27-31, 33-40, 43-48, 50-73, 75-77, 79-82, 85-89, 91-97, 99-104, 106-108, 110-112, 115, 117-124, 126-132, 134-141, 143-145, 147-174, 176-182, 184-190, 192-206, 208, 211, 213-215, 217-239, 241-243, 246-252, 256, 258-262, 264, 266-278, 280-286, 288-306, 308-318, 320-340, 342-346, 348-366, 368-375, 377-382, 384-389, 391-394, 396-400, 402-403, 406-410, 412-420, 423-440, 442-511, 513-521, 523-526, 528-531, 534-536, 538-543, 545-553, 556-562, 564-565, 568, 570-575, 577, 579-580, 582-594, 596-601, 603-604, 606-610, 612-614, 616-617, 619-620, 622-646, 648, 650-693, 695-707, 709-711, 713-723, 725-730, 732, 736-739, 741-751, 753-757, 759-777, 779-781, 783-784, 786-800, 802-803, 805, 807-811, 813-830, 833-834, 836-846, 848-856, 858-859, 861-862, 864-869, 871-880, 882-884, 886-893, 895-904, 906-911, 914-917, 919-924, 926, 928-950, 952-958, 961-972, 974-976, 978-982, 984-991, 993-998, 1000-1010, 1014-1023]
	stream status: stopped
	len: 129
	vb list dump: [4, 15, 22, 26, 32, 41-42, 49, 74, 78, 83-84, 90, 98, 105, 109, 113-114, 116, 125, 133, 142, 146, 175, 183, 191, 207, 209-210, 212, 216, 240, 244-245, 253-255, 257, 263, 265, 279, 287, 307, 319, 341, 347, 367, 376, 383, 390, 395, 401, 404-405, 411, 421-422, 441, 512, 522, 527, 532-533, 537, 544, 554-555, 563, 566-567, 569, 576, 578, 581, 595, 602, 605, 611, 615, 618, 621, 647, 649, 694, 708, 712, 724, 731, 733-735, 740, 752, 758, 778, 782, 785, 801, 804, 806, 812, 831-832, 835, 847, 857, 860, 863, 870, 881, 885, 894, 905, 912-913, 918, 925, 927, 951, 959-960, 973, 977, 983, 992, 999, 1011-1013]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: b6c55ce1fd5a59d4ceb255346dec6ee7
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 97
	v: [0-3, 5-14, 16-21, 23-25, 27-31, 33-40, 43-48, 50-73, 75-77, 79-82, 85-89, 91-97, 99-104, 106-108, 110-112]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 100
	v: [115, 117-124, 126-132, 134-141, 143-145, 147-174, 176-182, 184-190, 192-206, 208, 211, 213-215, 217-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 100
	v: [228-239, 241-243, 246-252, 256, 258-262, 264, 266-278, 280-286, 288-306, 308-318, 320-340]
Producer node: 172.31.2.134:9300	Node UUID: fcf7f0c6d9dacb0af9eb312009eb4dd6
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 97
	v: [911, 914-917, 919-924, 926, 928-950, 952-958, 961-972, 974-976, 978-982, 984-991, 993-998, 1000-1010, 1014-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 100
	v: [683-693, 695-707, 709-711, 713-723, 725-730, 732, 736-739, 741-751, 753-757, 759-777, 779-781, 783-784, 786-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 98
	v: [797-800, 802-803, 805, 807-811, 813-830, 833-834, 836-846, 848-856, 858-859, 861-862, 864-869, 871-880, 882-884, 886-893, 895-904, 906-910]
Producer node: 127.0.0.1:9303	Node UUID: 28208cb83453792b94127141277b33ab
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 101
	v: [456-511, 513-521, 523-526, 528-531, 534-536, 538-543, 545-553, 556-562, 564-565, 568]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 101
	v: [570-575, 577, 579-580, 582-594, 596-601, 603-604, 606-610, 612-614, 616-617, 619-620, 622-646, 648, 650-682]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 101
	v: [342-346, 348-366, 368-375, 377-382, 384-389, 391-394, 396-400, 402-403, 406-410, 412-420, 423-440, 442-455]


2021/12/25 05:03:29 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:03:34 eventingSetup.go:719: Waiting for function: TestBucketDeleteWithRebOut status to change to undeployed
2021/12/25 05:03:34 eventingSetup.go:727: Function: TestBucketDeleteWithRebOut status changed to undeployed
2021/12/25 05:03:39 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/25 05:03:39 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/25 05:03:39 eventingSetup.go:46: No eventing-con process running
2021/12/25 05:03:39 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut deleting in background"
}
2021/12/25 05:03:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:03:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:03:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:03:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:03:53 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/25 05:03:53 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 05:03:58 Rebalance progress: 100
--- PASS: TestBucketDeleteWithRebOut (462.26s)
=== RUN   TestN1QLRecursion
2021/12/25 05:04:00 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 05:04:01 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/25 05:04:01 Removing node(s): [] from the cluster
2021/12/25 05:04:06 Rebalance progress: 100
2021/12/25 05:04:12 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:04:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:04:24 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:04:29 Deploying app: TestN1QLRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:04:30 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
}
2021/12/25 05:04:30 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
} err 
2021/12/25 05:04:35 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:04:40 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:04:45 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:04:50 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:04:55 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:05:00 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/25 05:05:00 App: TestN1QLRecursion_function1 got deployed
2021/12/25 05:05:00 TestN1QLRecursion_function1 is deployed
2021/12/25 05:05:00 Deploying app: TestN1QLRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:05:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
}
2021/12/25 05:05:02 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
} err 
2021/12/25 05:05:07 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/25 05:05:12 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/25 05:05:17 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/25 05:05:22 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/25 05:05:22 App: TestN1QLRecursion_function3 got deployed
2021/12/25 05:05:22 TestN1QLRecursion_function3 is deployed
2021/12/25 05:05:22 Deploying app: TestN1QLRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:05:22 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
}
2021/12/25 05:05:22 response body {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
} err 
2021/12/25 05:05:22 Success: TestN1QLRecursion_function2 is not deployed
2021/12/25 05:05:27 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/25 05:05:32 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/25 05:05:37 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/25 05:05:42 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/25 05:05:42 App: TestN1QLRecursion_function1 got un-deployed
2021/12/25 05:05:42 No eventing-con process running
2021/12/25 05:05:42 Function: TestN1QLRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 deleting in background"
}
2021/12/25 05:05:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:05:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:05:51 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:05:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:06:01 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/25 05:06:06 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/25 05:06:11 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/25 05:06:16 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/25 05:06:16 App: TestN1QLRecursion_function3 got un-deployed
2021/12/25 05:06:16 No eventing-con process running
2021/12/25 05:06:16 Function: TestN1QLRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 deleting in background"
}
2021/12/25 05:06:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:06:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:06:25 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:06:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:06:31 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 05:06:36 Rebalance progress: 100
--- PASS: TestN1QLRecursion (155.50s)
=== RUN   TestN1QLAllowRecursion
2021/12/25 05:06:36 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 05:06:36 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/25 05:06:36 Removing node(s): [] from the cluster
2021/12/25 05:06:41 Rebalance progress: 100
2021/12/25 05:06:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:06:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:06:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:07:04 Deploying app: TestN1QLAllowRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:07:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
}
2021/12/25 05:07:05 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
} err 
2021/12/25 05:07:10 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/25 05:07:15 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/25 05:07:20 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/25 05:07:25 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/25 05:07:25 App: TestN1QLAllowRecursion_function1 got deployed
2021/12/25 05:07:25 TestN1QLAllowRecursion_function1 is deployed
2021/12/25 05:07:25 Deploying app: TestN1QLAllowRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:07:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
}
2021/12/25 05:07:27 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
} err 
2021/12/25 05:07:32 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/25 05:07:37 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/25 05:07:42 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/25 05:07:47 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/25 05:07:47 App: TestN1QLAllowRecursion_function3 got deployed
2021/12/25 05:07:47 TestN1QLAllowRecursion_function3 is deployed
2021/12/25 05:07:47 Deploying app: TestN1QLAllowRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:07:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
}
2021/12/25 05:07:48 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
} err 
2021/12/25 05:07:53 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/25 05:07:58 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/25 05:08:03 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/25 05:08:08 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/25 05:08:08 App: TestN1QLAllowRecursion_function2 got deployed
2021/12/25 05:08:08 Success: TestN1QLAllowRecursion_function2 is deployed after allow_interbucket_recursion
2021/12/25 05:08:13 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/25 05:08:18 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/25 05:08:23 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/25 05:08:28 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/25 05:08:28 App: TestN1QLAllowRecursion_function1 got un-deployed
2021/12/25 05:08:28 No eventing-con process running
2021/12/25 05:08:28 Function: TestN1QLAllowRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 deleting in background"
}
2021/12/25 05:08:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:08:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:08:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:08:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:08:47 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/25 05:08:52 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/25 05:08:57 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/25 05:09:02 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/25 05:09:02 App: TestN1QLAllowRecursion_function2 got un-deployed
2021/12/25 05:09:02 No eventing-con process running
2021/12/25 05:09:02 Function: TestN1QLAllowRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 deleting in background"
}
2021/12/25 05:09:04 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:09:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:09:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:09:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:09:21 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/25 05:09:26 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/25 05:09:31 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/25 05:09:36 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/25 05:09:36 App: TestN1QLAllowRecursion_function3 got un-deployed
2021/12/25 05:09:36 No eventing-con process running
2021/12/25 05:09:36 Function: TestN1QLAllowRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 deleting in background"
}
2021/12/25 05:09:38 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:09:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:09:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:09:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:09:52 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 05:09:57 Rebalance progress: 100
--- PASS: TestN1QLAllowRecursion (200.66s)
=== RUN   TestOnDeleteExpiryBucketOp
2021/12/25 05:09:57 Sleeping for some time
2021/12/25 05:10:07 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/25 05:10:07 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/25 05:10:07 No eventing-con process running
2021/12/25 05:10:07 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteExpiryBucketOp not found"
}
2021/12/25 05:10:08 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:10:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:10:15 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:10:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:10:20 Deploying app: TestOnDeleteExpiryBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:10:21 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp stored in temp store"
}
2021/12/25 05:10:21 Starting bucket ops to source bucket
2021/12/25 05:11:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 05:11:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 05:11:15 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 05:11:15 Starting bucket ops to source bucket
2021/12/25 05:11:17 Starting bucket ops to source bucket
2021/12/25 05:12:22 Waiting for dst bucket item count to get to: 7000 curr count: 5000
2021/12/25 05:12:27 Waiting for dst bucket item count to get to: 7000 curr count: 6641
2021/12/25 05:12:32 Waiting for dst bucket item count to get to: 7000 curr count: 6641
2021/12/25 05:12:37 Waiting for dst bucket item count to get to: 7000 curr count: 6641
2021/12/25 05:12:37 src & dst bucket item count matched up. src bucket count: 7000 dst bucket count: 7000
2021/12/25 05:12:37 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:12:43 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/25 05:12:48 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/25 05:12:53 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/25 05:12:58 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/25 05:12:58 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/25 05:12:58 No eventing-con process running
2021/12/25 05:12:58 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp deleting in background"
}
2021/12/25 05:13:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:13:10 Waiting for dst bucket item count to get to: 0 curr count: 499
2021/12/25 05:13:15 Waiting for dst bucket item count to get to: 0 curr count: 499
2021/12/25 05:13:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:13:16 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:13:26 Waiting for dst bucket item count to get to: 0 curr count: 7000
2021/12/25 05:13:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteExpiryBucketOp (209.84s)
=== RUN   TestTimerOverWriteSameReference
2021/12/25 05:13:31 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/25 05:13:31 App: TestTimerOverWriteSameReference got un-deployed
2021/12/25 05:13:32 No eventing-con process running
2021/12/25 05:13:32 Function: TestTimerOverWriteSameReference delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverWriteSameReference not found"
}
2021/12/25 05:13:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:13:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:13:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:13:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:13:45 Deploying app: TestTimerOverWriteSameReference. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:13:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference stored in temp store"
}
2021/12/25 05:13:51 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/25 05:13:56 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/25 05:14:01 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/25 05:14:06 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/25 05:14:06 App: TestTimerOverWriteSameReference got deployed
2021/12/25 05:14:16 Starting bucket ops to source bucket
2021/12/25 05:15:18 src & dst bucket item count matched up. src bucket count: 1 dst bucket count: 1
2021/12/25 05:15:28 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:15:33 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/25 05:15:38 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/25 05:15:43 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/25 05:15:48 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/25 05:15:48 App: TestTimerOverWriteSameReference got un-deployed
2021/12/25 05:15:48 No eventing-con process running
2021/12/25 05:15:48 Function: TestTimerOverWriteSameReference delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference deleting in background"
}
2021/12/25 05:15:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:16:00 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/25 05:16:05 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/25 05:16:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:16:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:16:16 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/25 05:16:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverWriteSameReference (169.90s)
=== RUN   TestCancelTimerBucketop
2021/12/25 05:16:21 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/25 05:16:21 App: TestCancelTimerBucketop got un-deployed
2021/12/25 05:16:21 No eventing-con process running
2021/12/25 05:16:21 Function: TestCancelTimerBucketop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCancelTimerBucketop not found"
}
2021/12/25 05:16:23 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:16:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:16:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:16:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:16:35 Deploying app: TestCancelTimerBucketop. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:16:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop stored in temp store"
}
2021/12/25 05:16:41 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/25 05:16:46 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/25 05:16:51 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/25 05:16:56 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/25 05:16:56 App: TestCancelTimerBucketop got deployed
2021/12/25 05:16:56 Starting bucket ops to source bucket
2021/12/25 05:17:03 Starting bucket ops to source bucket
2021/12/25 05:18:06 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/25 05:18:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:18:11 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/25 05:18:16 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/25 05:18:21 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/25 05:18:26 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/25 05:18:26 App: TestCancelTimerBucketop got un-deployed
2021/12/25 05:18:26 No eventing-con process running
2021/12/25 05:18:26 Function: TestCancelTimerBucketop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop deleting in background"
}
2021/12/25 05:18:28 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:18:38 Waiting for dst bucket item count to get to: 0 curr count: 535
2021/12/25 05:18:43 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/25 05:18:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:18:44 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:18:54 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:18:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCancelTimerBucketop (158.11s)
=== RUN   TestJSExpiryDate
2021/12/25 05:18:59 Starting bucket ops to source bucket
2021/12/25 05:19:13 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 05:19:13 Deploying app: TestJSExpiryDate. Item count src bucket: 5000 dst bucket: 0
2021/12/25 05:19:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate stored in temp store"
}
2021/12/25 05:19:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:19:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:19:49 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/25 05:19:54 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/25 05:19:59 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/25 05:20:04 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/25 05:20:04 App: TestJSExpiryDate got un-deployed
2021/12/25 05:20:04 No eventing-con process running
2021/12/25 05:20:04 Function: TestJSExpiryDate delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate deleting in background"
}
2021/12/25 05:20:06 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:20:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:20:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:20:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestJSExpiryDate (83.99s)
=== RUN   TestBinaryDoc
2021/12/25 05:20:23 Starting bucket ops to source bucket
2021/12/25 05:20:24 Deploying app: TestBinaryDoc. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:20:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc stored in temp store"
}
2021/12/25 05:20:25 Starting bucket ops to source bucket
2021/12/25 05:20:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:20:25 Deleting source bucket docs now
2021/12/25 05:20:25 Starting bucket ops to source bucket
2021/12/25 05:21:02 Waiting for dst bucket item count to get to: 32 curr count: 23
2021/12/25 05:21:07 Waiting for dst bucket item count to get to: 32 curr count: 28
2021/12/25 05:21:07 src & dst bucket item count matched up. src bucket count: 32 dst bucket count: 32
2021/12/25 05:21:07 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:21:12 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/25 05:21:17 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/25 05:21:22 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/25 05:21:27 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/25 05:21:27 App: TestBinaryDoc got un-deployed
2021/12/25 05:21:27 No eventing-con process running
2021/12/25 05:21:27 Function: TestBinaryDoc delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc deleting in background"
}
2021/12/25 05:21:29 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:21:39 Waiting for dst bucket item count to get to: 0 curr count: 32
2021/12/25 05:21:44 Waiting for dst bucket item count to get to: 0 curr count: 14
2021/12/25 05:21:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:21:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:21:56 Waiting for dst bucket item count to get to: 0 curr count: 32
2021/12/25 05:21:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBinaryDoc (97.41s)
=== RUN   TestConstantBindings
2021/12/25 05:21:56 Deploying app: TestConstantBindings. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:21:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings stored in temp store"
}
2021/12/25 05:21:57 Starting bucket ops to source bucket
2021/12/25 05:22:03 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/25 05:22:08 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/25 05:22:13 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/25 05:22:18 Waiting for dst bucket item count to get to: 6000 curr count: 902
2021/12/25 05:22:23 Waiting for dst bucket item count to get to: 6000 curr count: 902
2021/12/25 05:22:23 src & dst bucket item count matched up. src bucket count: 6000 dst bucket count: 6000
2021/12/25 05:22:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:22:28 Waiting for app: TestConstantBindings to get un-deployed
2021/12/25 05:22:33 Waiting for app: TestConstantBindings to get un-deployed
2021/12/25 05:22:38 Waiting for app: TestConstantBindings to get un-deployed
2021/12/25 05:22:43 Waiting for app: TestConstantBindings to get un-deployed
2021/12/25 05:22:43 App: TestConstantBindings got un-deployed
2021/12/25 05:22:43 No eventing-con process running
2021/12/25 05:22:43 Function: TestConstantBindings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings deleting in background"
}
2021/12/25 05:22:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:22:55 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:22:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:22:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:23:07 Waiting for dst bucket item count to get to: 0 curr count: 6000
2021/12/25 05:23:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestConstantBindings (70.63s)
=== RUN   TestCodeInsight
2021/12/25 05:23:12 Waiting for app: TestCodeInsight to get un-deployed
2021/12/25 05:23:12 App: TestCodeInsight got un-deployed
2021/12/25 05:23:12 No eventing-con process running
2021/12/25 05:23:12 Function: TestCodeInsight delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCodeInsight not found"
}
2021/12/25 05:23:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:23:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:23:20 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:23:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:23:25 Starting bucket ops to source bucket
2021/12/25 05:23:26 Deploying app: TestCodeInsight. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:23:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight stored in temp store"
}
2021/12/25 05:23:32 Waiting for app: TestCodeInsight to get deployed
2021/12/25 05:23:37 Waiting for app: TestCodeInsight to get deployed
2021/12/25 05:23:42 Waiting for app: TestCodeInsight to get deployed
2021/12/25 05:23:47 Waiting for app: TestCodeInsight to get deployed
2021/12/25 05:23:47 App: TestCodeInsight got deployed
2021/12/25 05:23:52 Waiting for dst bucket item count to get to: 1000 curr count: 195
2021/12/25 05:23:52 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/25 05:23:57 Waiting for app: TestCodeInsight to get un-deployed
2021/12/25 05:24:02 Waiting for app: TestCodeInsight to get un-deployed
2021/12/25 05:24:07 Waiting for app: TestCodeInsight to get un-deployed
2021/12/25 05:24:07 App: TestCodeInsight got un-deployed
2021/12/25 05:24:07 No eventing-con process running
2021/12/25 05:24:07 Function: TestCodeInsight delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight deleting in background"
}
2021/12/25 05:24:09 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:24:19 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:24:24 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/25 05:24:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:24:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:24:36 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:24:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCodeInsight (89.44s)
=== RUN   TestParserTransform
--- PASS: TestParserTransform (0.04s)
=== RUN   TestParserDetect
--- PASS: TestParserDetect (0.00s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	12987.523s

Functional tests took 3 hours, 36 minutes and 30 seconds.

N1ql tests

2021/12/25 05:24:47 Started n1ql
2021/12/25 05:24:55 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/25 05:24:56 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/25 05:24:56 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 05:24:56 Removing node(s): [] from the cluster
2021/12/25 05:25:01 Rebalance progress: 100
=== RUN   TestRecursiveMutationN1QL
2021/12/25 05:25:06 Waiting for app: TestRecursiveMutationN1QL to get un-deployed
2021/12/25 05:25:06 App: TestRecursiveMutationN1QL got un-deployed
2021/12/25 05:25:06 No eventing-con process running
2021/12/25 05:25:06 Function: TestRecursiveMutationN1QL delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestRecursiveMutationN1QL not found"
}
2021/12/25 05:25:08 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:25:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:25:14 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:25:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:25:19 Deploying app: TestRecursiveMutationN1QL. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:25:19 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Function: TestRecursiveMutationN1QL N1QL DML to source keyspace default:default"
}
--- PASS: TestRecursiveMutationN1QL (18.59s)
=== RUN   TestFlexReset1
2021/12/25 05:25:24 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/25 05:25:24 App: TestFlexReset1 got un-deployed
2021/12/25 05:25:24 No eventing-con process running
2021/12/25 05:25:24 Function: TestFlexReset1 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset1 not found"
}
2021/12/25 05:25:26 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:25:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:25:33 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:25:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:25:38 Starting bucket ops to source bucket
2021/12/25 05:25:39 Deploying app: TestFlexReset1. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:25:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 stored in temp store"
}
2021/12/25 05:25:45 Waiting for app: TestFlexReset1 to get deployed
2021/12/25 05:25:50 Waiting for app: TestFlexReset1 to get deployed
2021/12/25 05:25:55 Waiting for app: TestFlexReset1 to get deployed
2021/12/25 05:26:00 Waiting for app: TestFlexReset1 to get deployed
2021/12/25 05:26:00 App: TestFlexReset1 got deployed
2021/12/25 05:26:05 Waiting for dst bucket item count to get to: 1000 curr count: 500
2021/12/25 05:26:05 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/25 05:26:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:26:10 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/25 05:26:15 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/25 05:26:20 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/25 05:26:25 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/25 05:26:25 App: TestFlexReset1 got un-deployed
2021/12/25 05:26:25 No eventing-con process running
2021/12/25 05:26:25 Function: TestFlexReset1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 deleting in background"
}
2021/12/25 05:26:27 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:26:37 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:26:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:26:39 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:26:49 Waiting for dst bucket item count to get to: 0 curr count: 868
2021/12/25 05:26:54 Waiting for dst bucket item count to get to: 0 curr count: 368
2021/12/25 05:26:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset1 (94.17s)
=== RUN   TestFlexReset2
2021/12/25 05:26:59 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/25 05:26:59 App: TestFlexReset2 got un-deployed
2021/12/25 05:26:59 No eventing-con process running
2021/12/25 05:26:59 Function: TestFlexReset2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset2 not found"
}
2021/12/25 05:27:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:27:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:27:07 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:27:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:27:12 Starting bucket ops to source bucket
2021/12/25 05:27:13 Deploying app: TestFlexReset2. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:27:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 stored in temp store"
}
2021/12/25 05:27:19 Waiting for app: TestFlexReset2 to get deployed
2021/12/25 05:27:24 Waiting for app: TestFlexReset2 to get deployed
2021/12/25 05:27:29 Waiting for app: TestFlexReset2 to get deployed
2021/12/25 05:27:34 Waiting for app: TestFlexReset2 to get deployed
2021/12/25 05:27:34 App: TestFlexReset2 got deployed
2021/12/25 05:27:39 Waiting for dst bucket item count to get to: 1000 curr count: 634
2021/12/25 05:27:39 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/25 05:27:39 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:27:44 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/25 05:27:49 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/25 05:27:54 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/25 05:27:59 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/25 05:27:59 App: TestFlexReset2 got un-deployed
2021/12/25 05:27:59 No eventing-con process running
2021/12/25 05:27:59 Function: TestFlexReset2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 deleting in background"
}
2021/12/25 05:28:01 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:28:11 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:28:16 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/25 05:28:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:28:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:28:28 Waiting for dst bucket item count to get to: 0 curr count: 654
2021/12/25 05:28:33 Waiting for dst bucket item count to get to: 0 curr count: 154
2021/12/25 05:28:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset2 (99.39s)
=== RUN   TestN1QLLabelledBreak
2021/12/25 05:28:38 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/25 05:28:38 App: TestN1QLLabelledBreak got un-deployed
2021/12/25 05:28:38 No eventing-con process running
2021/12/25 05:28:38 Function: TestN1QLLabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLLabelledBreak not found"
}
2021/12/25 05:28:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:28:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:28:46 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:28:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:28:51 Deploying app: TestN1QLLabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:28:53 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak stored in temp store"
}
2021/12/25 05:28:58 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/25 05:29:03 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/25 05:29:08 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/25 05:29:13 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/25 05:29:13 App: TestN1QLLabelledBreak got deployed
2021/12/25 05:29:13 Starting bucket ops to source bucket
2021/12/25 05:29:22 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/25 05:29:27 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/25 05:29:32 Waiting for dst bucket item count to get to: 10000 curr count: 1045
2021/12/25 05:29:37 Waiting for dst bucket item count to get to: 10000 curr count: 1045
2021/12/25 05:29:42 Waiting for dst bucket item count to get to: 10000 curr count: 5084
2021/12/25 05:29:47 Waiting for dst bucket item count to get to: 10000 curr count: 7128
2021/12/25 05:29:52 Waiting for dst bucket item count to get to: 10000 curr count: 9150
2021/12/25 05:29:57 Waiting for dst bucket item count to get to: 10000 curr count: 9779
2021/12/25 05:29:57 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 05:29:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:30:02 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/25 05:30:07 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/25 05:30:12 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/25 05:30:17 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/25 05:30:17 App: TestN1QLLabelledBreak got un-deployed
2021/12/25 05:30:17 No eventing-con process running
2021/12/25 05:30:17 Function: TestN1QLLabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak deleting in background"
}
2021/12/25 05:30:19 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:30:29 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:30:34 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/25 05:30:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:30:36 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:30:46 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 05:30:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLLabelledBreak (133.21s)
=== RUN   TestN1QLUnlabelledBreak
2021/12/25 05:30:51 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/25 05:30:51 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/25 05:30:51 No eventing-con process running
2021/12/25 05:30:51 Function: TestN1QLUnlabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLUnlabelledBreak not found"
}
2021/12/25 05:30:53 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:30:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:31:00 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:31:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:31:05 Deploying app: TestN1QLUnlabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:31:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak stored in temp store"
}
2021/12/25 05:31:11 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/25 05:31:16 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/25 05:31:21 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/25 05:31:26 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/25 05:31:26 App: TestN1QLUnlabelledBreak got deployed
2021/12/25 05:31:26 Starting bucket ops to source bucket
2021/12/25 05:31:36 Waiting for dst bucket item count to get to: 10000 curr count: 459
2021/12/25 05:31:41 Waiting for dst bucket item count to get to: 10000 curr count: 459
2021/12/25 05:31:46 Waiting for dst bucket item count to get to: 10000 curr count: 459
2021/12/25 05:31:51 Waiting for dst bucket item count to get to: 10000 curr count: 5402
2021/12/25 05:31:56 Waiting for dst bucket item count to get to: 10000 curr count: 5402
2021/12/25 05:31:56 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 05:31:56 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:32:01 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/25 05:32:06 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/25 05:32:11 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/25 05:32:16 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/25 05:32:16 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/25 05:32:16 No eventing-con process running
2021/12/25 05:32:16 Function: TestN1QLUnlabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak deleting in background"
}
2021/12/25 05:32:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:32:28 Waiting for dst bucket item count to get to: 0 curr count: 2622
2021/12/25 05:32:33 Waiting for dst bucket item count to get to: 0 curr count: 122
2021/12/25 05:32:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:32:34 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:32:44 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/25 05:32:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLUnlabelledBreak (118.28s)
=== RUN   TestN1QLThrowStatement
2021/12/25 05:32:49 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/25 05:32:49 App: TestN1QLThrowStatement got un-deployed
2021/12/25 05:32:49 No eventing-con process running
2021/12/25 05:32:49 Function: TestN1QLThrowStatement delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLThrowStatement not found"
}
2021/12/25 05:32:51 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:32:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:32:58 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:33:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:33:03 Deploying app: TestN1QLThrowStatement. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:33:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement stored in temp store"
}
2021/12/25 05:33:09 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/25 05:33:14 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/25 05:33:19 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/25 05:33:24 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/25 05:33:24 App: TestN1QLThrowStatement got deployed
2021/12/25 05:33:24 Starting bucket ops to source bucket
2021/12/25 05:33:34 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/25 05:33:39 Waiting for dst bucket item count to get to: 10000 curr count: 202
2021/12/25 05:33:44 Waiting for dst bucket item count to get to: 10000 curr count: 644
2021/12/25 05:33:49 Waiting for dst bucket item count to get to: 10000 curr count: 2470
2021/12/25 05:33:54 Waiting for dst bucket item count to get to: 10000 curr count: 4505
2021/12/25 05:33:59 Waiting for dst bucket item count to get to: 10000 curr count: 6580
2021/12/25 05:34:04 Waiting for dst bucket item count to get to: 10000 curr count: 8617
2021/12/25 05:34:09 Waiting for dst bucket item count to get to: 10000 curr count: 9514
2021/12/25 05:34:09 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/25 05:34:09 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:34:14 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/25 05:34:19 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/25 05:34:24 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/25 05:34:29 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/25 05:34:29 App: TestN1QLThrowStatement got un-deployed
2021/12/25 05:34:29 No eventing-con process running
2021/12/25 05:34:29 Function: TestN1QLThrowStatement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement deleting in background"
}
2021/12/25 05:34:31 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:34:41 Waiting for dst bucket item count to get to: 0 curr count: 4691
2021/12/25 05:34:46 Waiting for dst bucket item count to get to: 0 curr count: 4691
2021/12/25 05:34:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:34:48 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:34:58 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:34:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLThrowStatement (133.15s)
=== RUN   TestN1QLNestedForLoop
2021/12/25 05:35:03 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/25 05:35:03 App: TestN1QLNestedForLoop got un-deployed
2021/12/25 05:35:03 No eventing-con process running
2021/12/25 05:35:03 Function: TestN1QLNestedForLoop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLNestedForLoop not found"
}
2021/12/25 05:35:04 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:35:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:35:11 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:35:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:35:16 Deploying app: TestN1QLNestedForLoop. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:35:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop stored in temp store"
}
2021/12/25 05:35:22 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/25 05:35:27 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/25 05:35:32 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/25 05:35:37 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/25 05:35:37 App: TestN1QLNestedForLoop got deployed
2021/12/25 05:35:37 Starting bucket ops to source bucket
2021/12/25 05:35:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 05:35:52 Waiting for dst bucket item count to get to: 5000 curr count: 64
2021/12/25 05:35:57 Waiting for dst bucket item count to get to: 5000 curr count: 64
2021/12/25 05:36:02 Waiting for dst bucket item count to get to: 5000 curr count: 1890
2021/12/25 05:36:07 Waiting for dst bucket item count to get to: 5000 curr count: 1890
2021/12/25 05:36:12 Waiting for dst bucket item count to get to: 5000 curr count: 4308
2021/12/25 05:36:17 Waiting for dst bucket item count to get to: 5000 curr count: 4308
2021/12/25 05:36:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 05:36:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:36:22 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/25 05:36:27 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/25 05:36:32 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/25 05:36:37 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/25 05:36:37 App: TestN1QLNestedForLoop got un-deployed
2021/12/25 05:36:37 No eventing-con process running
2021/12/25 05:36:37 Function: TestN1QLNestedForLoop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop deleting in background"
}
2021/12/25 05:36:39 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:36:49 Waiting for dst bucket item count to get to: 0 curr count: 4630
2021/12/25 05:36:54 Waiting for dst bucket item count to get to: 0 curr count: 2130
2021/12/25 05:36:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:36:55 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:37:05 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:37:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLNestedForLoop (127.88s)
=== RUN   TestN1QLAutoClose
2021/12/25 05:37:10 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/25 05:37:10 App: TestN1QLAutoClose got un-deployed
2021/12/25 05:37:11 No eventing-con process running
2021/12/25 05:37:11 Function: TestN1QLAutoClose delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLAutoClose not found"
}
2021/12/25 05:37:12 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:37:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:37:19 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:37:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:37:24 Deploying app: TestN1QLAutoClose. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:37:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose stored in temp store"
}
2021/12/25 05:37:30 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/25 05:37:35 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/25 05:37:40 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/25 05:37:45 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/25 05:37:45 App: TestN1QLAutoClose got deployed
2021/12/25 05:37:45 Starting bucket ops to source bucket
2021/12/25 05:37:51 Waiting for dst bucket item count to get to: 500 curr count: 0
2021/12/25 05:37:56 Waiting for dst bucket item count to get to: 500 curr count: 5
2021/12/25 05:38:01 Waiting for dst bucket item count to get to: 500 curr count: 5
2021/12/25 05:38:06 Waiting for dst bucket item count to get to: 500 curr count: 28
2021/12/25 05:38:11 Waiting for dst bucket item count to get to: 500 curr count: 28
2021/12/25 05:38:16 Waiting for dst bucket item count to get to: 500 curr count: 49
2021/12/25 05:38:21 Waiting for dst bucket item count to get to: 500 curr count: 49
2021/12/25 05:38:26 Waiting for dst bucket item count to get to: 500 curr count: 72
2021/12/25 05:38:31 Waiting for dst bucket item count to get to: 500 curr count: 72
2021/12/25 05:38:36 Waiting for dst bucket item count to get to: 500 curr count: 92
2021/12/25 05:38:41 Waiting for dst bucket item count to get to: 500 curr count: 92
2021/12/25 05:38:46 Waiting for dst bucket item count to get to: 500 curr count: 114
2021/12/25 05:38:51 Waiting for dst bucket item count to get to: 500 curr count: 114
2021/12/25 05:38:56 Waiting for dst bucket item count to get to: 500 curr count: 114
2021/12/25 05:39:01 Waiting for dst bucket item count to get to: 500 curr count: 135
2021/12/25 05:39:06 Waiting for dst bucket item count to get to: 500 curr count: 135
2021/12/25 05:39:11 Waiting for dst bucket item count to get to: 500 curr count: 156
2021/12/25 05:39:16 Waiting for dst bucket item count to get to: 500 curr count: 156
2021/12/25 05:39:21 Waiting for dst bucket item count to get to: 500 curr count: 179
2021/12/25 05:39:26 Waiting for dst bucket item count to get to: 500 curr count: 179
2021/12/25 05:39:31 Waiting for dst bucket item count to get to: 500 curr count: 201
2021/12/25 05:39:36 Waiting for dst bucket item count to get to: 500 curr count: 201
2021/12/25 05:39:41 Waiting for dst bucket item count to get to: 500 curr count: 222
2021/12/25 05:39:46 Waiting for dst bucket item count to get to: 500 curr count: 222
2021/12/25 05:39:51 Waiting for dst bucket item count to get to: 500 curr count: 243
2021/12/25 05:39:56 Waiting for dst bucket item count to get to: 500 curr count: 243
2021/12/25 05:40:01 Waiting for dst bucket item count to get to: 500 curr count: 264
2021/12/25 05:40:06 Waiting for dst bucket item count to get to: 500 curr count: 264
2021/12/25 05:40:11 Waiting for dst bucket item count to get to: 500 curr count: 283
2021/12/25 05:40:16 Waiting for dst bucket item count to get to: 500 curr count: 297
2021/12/25 05:40:21 Waiting for dst bucket item count to get to: 500 curr count: 306
2021/12/25 05:40:26 Waiting for dst bucket item count to get to: 500 curr count: 318
2021/12/25 05:40:31 Waiting for dst bucket item count to get to: 500 curr count: 326
2021/12/25 05:40:36 Waiting for dst bucket item count to get to: 500 curr count: 337
2021/12/25 05:40:41 Waiting for dst bucket item count to get to: 500 curr count: 347
2021/12/25 05:40:46 Waiting for dst bucket item count to get to: 500 curr count: 359
2021/12/25 05:40:51 Waiting for dst bucket item count to get to: 500 curr count: 368
2021/12/25 05:40:56 Waiting for dst bucket item count to get to: 500 curr count: 377
2021/12/25 05:41:01 Waiting for dst bucket item count to get to: 500 curr count: 389
2021/12/25 05:41:06 Waiting for dst bucket item count to get to: 500 curr count: 401
2021/12/25 05:41:11 Waiting for dst bucket item count to get to: 500 curr count: 410
2021/12/25 05:41:16 Waiting for dst bucket item count to get to: 500 curr count: 423
2021/12/25 05:41:21 Waiting for dst bucket item count to get to: 500 curr count: 433
2021/12/25 05:41:26 Waiting for dst bucket item count to get to: 500 curr count: 444
2021/12/25 05:41:31 Waiting for dst bucket item count to get to: 500 curr count: 454
2021/12/25 05:41:36 Waiting for dst bucket item count to get to: 500 curr count: 467
2021/12/25 05:41:41 Waiting for dst bucket item count to get to: 500 curr count: 477
2021/12/25 05:41:46 Waiting for dst bucket item count to get to: 500 curr count: 497
2021/12/25 05:41:51 Waiting for dst bucket item count to get to: 500 curr count: 497
2021/12/25 05:41:51 src & dst bucket item count matched up. src bucket count: 500 dst bucket count: 500
2021/12/25 05:41:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:41:56 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/25 05:42:01 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/25 05:42:07 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/25 05:42:12 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/25 05:42:12 App: TestN1QLAutoClose got un-deployed
2021/12/25 05:42:12 No eventing-con process running
2021/12/25 05:42:12 Function: TestN1QLAutoClose delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose deleting in background"
}
2021/12/25 05:42:13 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:42:23 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/25 05:42:28 Waiting for dst bucket item count to get to: 0 curr count: 250
2021/12/25 05:42:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:42:30 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:42:40 Waiting for dst bucket item count to get to: 0 curr count: 348
2021/12/25 05:42:45 Waiting for dst bucket item count to get to: 0 curr count: 348
2021/12/25 05:42:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLAutoClose (339.66s)
=== RUN   TestN1QLPosParams
2021/12/25 05:42:50 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/25 05:42:50 App: TestN1QLPosParams got un-deployed
2021/12/25 05:42:50 No eventing-con process running
2021/12/25 05:42:50 Function: TestN1QLPosParams delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLPosParams not found"
}
2021/12/25 05:42:52 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:42:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:42:59 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:43:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:43:04 Deploying app: TestN1QLPosParams. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:43:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams stored in temp store"
}
2021/12/25 05:43:10 Waiting for app: TestN1QLPosParams to get deployed
2021/12/25 05:43:15 Waiting for app: TestN1QLPosParams to get deployed
2021/12/25 05:43:20 Waiting for app: TestN1QLPosParams to get deployed
2021/12/25 05:43:25 Waiting for app: TestN1QLPosParams to get deployed
2021/12/25 05:43:25 App: TestN1QLPosParams got deployed
2021/12/25 05:43:25 Starting bucket ops to source bucket
2021/12/25 05:43:35 Waiting for dst bucket item count to get to: 5000 curr count: 354
2021/12/25 05:43:40 Waiting for dst bucket item count to get to: 5000 curr count: 1073
2021/12/25 05:43:45 Waiting for dst bucket item count to get to: 5000 curr count: 3219
2021/12/25 05:43:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 05:43:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:43:50 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/25 05:43:55 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/25 05:44:00 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/25 05:44:05 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/25 05:44:05 App: TestN1QLPosParams got un-deployed
2021/12/25 05:44:05 No eventing-con process running
2021/12/25 05:44:05 Function: TestN1QLPosParams delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams deleting in background"
}
2021/12/25 05:44:07 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:44:17 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:44:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:44:18 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:44:28 Waiting for dst bucket item count to get to: 0 curr count: 4254
2021/12/25 05:44:33 Waiting for dst bucket item count to get to: 0 curr count: 1754
2021/12/25 05:44:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLPosParams (108.28s)
=== RUN   TestN1QLExhaustConnPool
2021/12/25 05:44:38 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/25 05:44:38 App: TestN1QLExhaustConnPool got un-deployed
2021/12/25 05:44:38 No eventing-con process running
2021/12/25 05:44:38 Function: TestN1QLExhaustConnPool delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLExhaustConnPool not found"
}
2021/12/25 05:44:40 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:44:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:44:47 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:44:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:44:52 Deploying app: TestN1QLExhaustConnPool. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:44:53 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool stored in temp store"
}
2021/12/25 05:44:58 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/25 05:45:03 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/25 05:45:08 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/25 05:45:13 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/25 05:45:13 App: TestN1QLExhaustConnPool got deployed
2021/12/25 05:45:13 Starting bucket ops to source bucket
2021/12/25 05:45:18 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/25 05:45:23 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/25 05:45:23 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/25 05:45:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:45:29 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/25 05:45:34 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/25 05:45:39 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/25 05:45:44 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/25 05:45:44 App: TestN1QLExhaustConnPool got un-deployed
2021/12/25 05:45:44 No eventing-con process running
2021/12/25 05:45:44 Function: TestN1QLExhaustConnPool delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool deleting in background"
}
2021/12/25 05:45:45 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:45:55 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 05:45:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:45:57 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:46:07 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/25 05:46:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLExhaustConnPool (93.82s)
=== RUN   TestN1QLGraceSufficientTimeOut
2021/12/25 05:46:12 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/25 05:46:12 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/25 05:46:12 No eventing-con process running
2021/12/25 05:46:12 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLGraceSufficientTimeOut not found"
}
2021/12/25 05:46:14 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:46:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:46:21 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:46:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:46:26 Starting bucket ops to source bucket
2021/12/25 05:46:43 Starting bucket ops to source bucket
2021/12/25 05:47:00 Deploying app: TestN1QLGraceSufficientTimeOut. Item count src bucket: 30000 dst bucket: 9255
2021/12/25 05:47:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut stored in temp store"
}
2021/12/25 05:47:06 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/25 05:47:11 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/25 05:47:16 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/25 05:47:21 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/25 05:47:21 App: TestN1QLGraceSufficientTimeOut got deployed
2021/12/25 05:47:21 Starting bucket ops to source bucket
2021/12/25 05:47:26 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:31 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:36 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:41 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:46 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:51 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:47:56 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:48:01 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:48:06 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:48:11 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/25 05:48:11 src & dst bucket item count matched up. src bucket count: 30001 dst bucket count: 30001
2021/12/25 05:48:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:48:17 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/25 05:48:22 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/25 05:48:27 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/25 05:48:32 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/25 05:48:32 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/25 05:48:32 No eventing-con process running
2021/12/25 05:48:32 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut deleting in background"
}
2021/12/25 05:48:34 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:48:44 Waiting for dst bucket item count to get to: 0 curr count: 30000
2021/12/25 05:48:49 Waiting for dst bucket item count to get to: 0 curr count: 15000
2021/12/25 05:48:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:48:50 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:49:00 Waiting for dst bucket item count to get to: 0 curr count: 24493
2021/12/25 05:49:05 Waiting for dst bucket item count to get to: 0 curr count: 24493
2021/12/25 05:49:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLGraceSufficientTimeOut (178.21s)
=== RUN   TestN1QLQueryConsistency
2021/12/25 05:49:10 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/25 05:49:10 App: n1qlquery_consistency got un-deployed
2021/12/25 05:49:10 No eventing-con process running
2021/12/25 05:49:10 Function: n1qlquery_consistency delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: n1qlquery_consistency not found"
}
2021/12/25 05:49:12 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:49:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:49:19 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:49:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:49:24 Starting bucket ops to source bucket
2021/12/25 05:49:27 Deploying app: n1qlquery_consistency. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:49:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency stored in temp store"
}
2021/12/25 05:49:33 Waiting for app: n1qlquery_consistency to get deployed
2021/12/25 05:49:38 Waiting for app: n1qlquery_consistency to get deployed
2021/12/25 05:49:43 Waiting for app: n1qlquery_consistency to get deployed
2021/12/25 05:49:48 Waiting for app: n1qlquery_consistency to get deployed
2021/12/25 05:49:48 App: n1qlquery_consistency got deployed
2021/12/25 05:49:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/25 05:49:58 Waiting for dst bucket item count to get to: 5000 curr count: 4619
2021/12/25 05:50:03 Waiting for dst bucket item count to get to: 5000 curr count: 4619
2021/12/25 05:50:03 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/25 05:50:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/25 05:50:08 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/25 05:50:14 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/25 05:50:19 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/25 05:50:24 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/25 05:50:24 App: n1qlquery_consistency got un-deployed
2021/12/25 05:50:24 No eventing-con process running
2021/12/25 05:50:24 Function: n1qlquery_consistency delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency deleting in background"
}
2021/12/25 05:50:25 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:50:35 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:50:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:50:37 Context: Bucket flush, response code: 200 dump: 
2021/12/25 05:50:47 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/25 05:50:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLQueryConsistency (101.74s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	1557.038s

N1ql tests took 0 hours, 26 minutes and 0 seconds.

Starting server
Starting server: attempt 1

Rebalance tests

2021/12/25 05:52:19 Started rebalance
=== RUN   TestDuplicateWithEventingRebIn
2021/12/25 05:52:37 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
running apps fetch error json: cannot unmarshal number into Go value of type string
2021/12/25 05:52:42 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/25 05:52:42 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/25 05:52:42 No eventing-con process running
2021/12/25 05:52:42 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebIn not found"
}
2021/12/25 05:52:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:52:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:52:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:52:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:52:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:52:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:53:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:53:00 Deploying app: TestDuplicateWithEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:53:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn stored in temp store"
}
2021/12/25 05:53:06 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/25 05:53:11 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/25 05:53:16 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/25 05:53:21 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/25 05:53:21 App: TestDuplicateWithEventingRebIn got deployed
2021/12/25 05:53:21 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 05:53:21 Starting bucket ops to source bucket
2021/12/25 05:53:36 Removing node(s): [] from the cluster
2021/12/25 05:53:41 Rebalance progress: 0
2021/12/25 05:53:46 Rebalance progress: 0
2021/12/25 05:53:51 Rebalance progress: 25
2021/12/25 05:53:56 Rebalance progress: 25
2021/12/25 05:54:01 Rebalance progress: 25
2021/12/25 05:54:06 Rebalance progress: 98.68421052631578
2021/12/25 05:54:11 Rebalance progress: 98.68421052631578
2021/12/25 05:54:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:54:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:54:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:54:16 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/25 05:54:16 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 05:54:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 05:54:21 Rebalance progress: 25
2021/12/25 05:54:26 Rebalance progress: 25
2021/12/25 05:54:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:54:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:54:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:54:31 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/25 05:54:31 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 05:54:31 Waiting for src bucket item count to get to: 200000 curr count: 93871
2021/12/25 05:54:36 Waiting for src bucket item count to get to: 200000 curr count: 110753
2021/12/25 05:54:41 Waiting for src bucket item count to get to: 200000 curr count: 110753
2021/12/25 05:54:46 Waiting for src bucket item count to get to: 200000 curr count: 127261
2021/12/25 05:54:51 Waiting for src bucket item count to get to: 200000 curr count: 127261
2021/12/25 05:54:56 Waiting for src bucket item count to get to: 200000 curr count: 144193
2021/12/25 05:55:01 Waiting for src bucket item count to get to: 200000 curr count: 144193
2021/12/25 05:55:06 Waiting for src bucket item count to get to: 200000 curr count: 160717
2021/12/25 05:55:11 Waiting for src bucket item count to get to: 200000 curr count: 160717
2021/12/25 05:55:16 Waiting for src bucket item count to get to: 200000 curr count: 177125
2021/12/25 05:55:21 Waiting for src bucket item count to get to: 200000 curr count: 177125
2021/12/25 05:55:26 Waiting for src bucket item count to get to: 200000 curr count: 193917
2021/12/25 05:55:31 Waiting for src bucket item count to get to: 200000 curr count: 193917
2021/12/25 05:55:41 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:55:46 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:55:51 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:55:56 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:01 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:06 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:11 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:16 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:21 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:26 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/25 05:56:26 Post SET ops. Expected item count: 200000 got 200001
2021/12/25 05:56:31 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/25 05:56:36 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/25 05:56:41 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/25 05:56:46 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/25 05:56:46 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/25 05:56:46 No eventing-con process running
2021/12/25 05:56:46 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn deleting in background"
}
2021/12/25 05:56:48 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:56:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:56:58 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 05:56:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:57:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:57:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:57:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebIn (277.96s)
=== RUN   TestDuplicateWithEventingRebOut
2021/12/25 05:57:10 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 05:57:10 Removing node(s): [] from the cluster
2021/12/25 05:57:15 Rebalance progress: 100
2021/12/25 05:57:20 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/25 05:57:20 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/25 05:57:20 No eventing-con process running
2021/12/25 05:57:20 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebOut not found"
}
2021/12/25 05:57:22 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:57:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:57:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 05:57:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:57:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:57:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:57:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 05:57:39 Deploying app: TestDuplicateWithEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 05:57:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut stored in temp store"
}
2021/12/25 05:57:45 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/25 05:57:50 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/25 05:57:55 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/25 05:58:00 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/25 05:58:00 App: TestDuplicateWithEventingRebOut got deployed
2021/12/25 05:58:00 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 05:58:00 Starting bucket ops to source bucket
2021/12/25 05:58:05 Rebalance progress: 25
2021/12/25 05:58:10 Rebalance progress: 25
2021/12/25 05:58:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:58:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 05:58:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 05:58:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/25 05:58:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 05:58:15 Waiting for src bucket item count to get to: 200000 curr count: 14880
2021/12/25 05:58:20 Waiting for src bucket item count to get to: 200000 curr count: 31804
2021/12/25 05:58:25 Waiting for src bucket item count to get to: 200000 curr count: 31804
2021/12/25 05:58:30 Waiting for src bucket item count to get to: 200000 curr count: 48510
2021/12/25 05:58:35 Waiting for src bucket item count to get to: 200000 curr count: 48510
2021/12/25 05:58:40 Waiting for src bucket item count to get to: 200000 curr count: 65350
2021/12/25 05:58:45 Waiting for src bucket item count to get to: 200000 curr count: 65350
2021/12/25 05:58:50 Waiting for src bucket item count to get to: 200000 curr count: 65350
2021/12/25 05:58:55 Waiting for src bucket item count to get to: 200000 curr count: 81857
2021/12/25 05:59:00 Waiting for src bucket item count to get to: 200000 curr count: 81857
2021/12/25 05:59:05 Waiting for src bucket item count to get to: 200000 curr count: 98510
2021/12/25 05:59:10 Waiting for src bucket item count to get to: 200000 curr count: 98510
2021/12/25 05:59:15 Waiting for src bucket item count to get to: 200000 curr count: 115261
2021/12/25 05:59:20 Waiting for src bucket item count to get to: 200000 curr count: 115261
2021/12/25 05:59:25 Waiting for src bucket item count to get to: 200000 curr count: 132141
2021/12/25 05:59:30 Waiting for src bucket item count to get to: 200000 curr count: 132141
2021/12/25 05:59:35 Waiting for src bucket item count to get to: 200000 curr count: 149022
2021/12/25 05:59:40 Waiting for src bucket item count to get to: 200000 curr count: 149022
2021/12/25 05:59:45 Waiting for src bucket item count to get to: 200000 curr count: 165877
2021/12/25 05:59:50 Waiting for src bucket item count to get to: 200000 curr count: 165877
2021/12/25 05:59:55 Waiting for src bucket item count to get to: 200000 curr count: 182542
2021/12/25 06:00:00 Waiting for src bucket item count to get to: 200000 curr count: 182542
2021/12/25 06:00:05 Waiting for src bucket item count to get to: 200000 curr count: 199067
2021/12/25 06:00:10 Waiting for src bucket item count to get to: 200000 curr count: 199067
2021/12/25 06:00:15 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:00:15 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:00:20 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/25 06:00:25 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/25 06:00:30 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/25 06:00:35 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/25 06:00:35 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/25 06:00:35 No eventing-con process running
2021/12/25 06:00:35 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut deleting in background"
}
2021/12/25 06:00:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:00:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:00:47 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:00:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:00:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:00:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:00:59 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:00:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebOut (233.98s)
=== RUN   TestDuplicateWithEventingSwapReb
2021/12/25 06:01:09 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/25 06:01:09 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/25 06:01:09 No eventing-con process running
2021/12/25 06:01:09 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingSwapReb not found"
}
2021/12/25 06:01:11 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:01:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:01:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:01:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:01:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:01:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:01:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:01:27 Deploying app: TestDuplicateWithEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:01:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb stored in temp store"
}
2021/12/25 06:01:33 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/25 06:01:38 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/25 06:01:43 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/25 06:01:48 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/25 06:01:48 App: TestDuplicateWithEventingSwapReb got deployed
2021/12/25 06:01:48 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:01:48 Starting bucket ops to source bucket
2021/12/25 06:01:49 Removing node(s): [] from the cluster
2021/12/25 06:01:54 Rebalance progress: 25
2021/12/25 06:01:59 Rebalance progress: 25
2021/12/25 06:02:04 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:02:04 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/25 06:02:04 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:02:04 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:02:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:02:16 Rebalance progress: 16.66666666666667
2021/12/25 06:02:21 Rebalance progress: 16.66666666666667
2021/12/25 06:02:26 Rebalance progress: 16.66666666666667
2021/12/25 06:02:31 Rebalance progress: 27.03862660944206
2021/12/25 06:02:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:02:36 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:02:36 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:02:41 Rebalance progress: 16.66666666666667
2021/12/25 06:02:46 Rebalance progress: 16.66666666666667
2021/12/25 06:02:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:02:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:02:51 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:02:51 Waiting for src bucket item count to get to: 200000 curr count: 76460
2021/12/25 06:02:56 Waiting for src bucket item count to get to: 200000 curr count: 93056
2021/12/25 06:03:01 Waiting for src bucket item count to get to: 200000 curr count: 93056
2021/12/25 06:03:06 Waiting for src bucket item count to get to: 200000 curr count: 109544
2021/12/25 06:03:11 Waiting for src bucket item count to get to: 200000 curr count: 109544
2021/12/25 06:03:16 Waiting for src bucket item count to get to: 200000 curr count: 126280
2021/12/25 06:03:21 Waiting for src bucket item count to get to: 200000 curr count: 126280
2021/12/25 06:03:26 Waiting for src bucket item count to get to: 200000 curr count: 142618
2021/12/25 06:03:31 Waiting for src bucket item count to get to: 200000 curr count: 142618
2021/12/25 06:03:36 Waiting for src bucket item count to get to: 200000 curr count: 158933
2021/12/25 06:03:41 Waiting for src bucket item count to get to: 200000 curr count: 175149
2021/12/25 06:03:46 Waiting for src bucket item count to get to: 200000 curr count: 175149
2021/12/25 06:03:51 Waiting for src bucket item count to get to: 200000 curr count: 191225
2021/12/25 06:03:56 Waiting for src bucket item count to get to: 200000 curr count: 191225
2021/12/25 06:04:01 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:04:01 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:04:06 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/25 06:04:11 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/25 06:04:16 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/25 06:04:21 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/25 06:04:21 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/25 06:04:21 No eventing-con process running
2021/12/25 06:04:22 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb deleting in background"
}
2021/12/25 06:04:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:04:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:04:33 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:04:38 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:04:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:04:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:04:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:04:50 Waiting for dst bucket item count to get to: 0 curr count: 15300
2021/12/25 06:04:55 Waiting for dst bucket item count to get to: 0 curr count: 15300
2021/12/25 06:04:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingSwapReb (236.11s)
=== RUN   TestDuplicateWithKVRebIn
2021/12/25 06:05:05 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/25 06:05:05 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/25 06:05:05 No eventing-con process running
2021/12/25 06:05:05 Function: TestDuplicateWithKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebIn not found"
}
2021/12/25 06:05:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:05:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:05:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:05:13 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:05:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:05:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:05:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:05:23 Deploying app: TestDuplicateWithKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:05:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn stored in temp store"
}
2021/12/25 06:05:30 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/25 06:05:35 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/25 06:05:40 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/25 06:05:45 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/25 06:05:45 App: TestDuplicateWithKVRebIn got deployed
2021/12/25 06:05:45 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 06:05:45 Starting bucket ops to source bucket
2021/12/25 06:05:45 Removing node(s): [] from the cluster
2021/12/25 06:05:50 Rebalance progress: 16.66666666666667
2021/12/25 06:05:55 Rebalance progress: 16.66666666666667
2021/12/25 06:06:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:06:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:06:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:06:00 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:06:00 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:06:05 Rebalance progress: 16.66666666666667
2021/12/25 06:06:10 Rebalance progress: 16.66666666666667
2021/12/25 06:06:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:06:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:06:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:06:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:06:15 Waiting for src bucket item count to get to: 200000 curr count: 39245
2021/12/25 06:06:20 Waiting for src bucket item count to get to: 200000 curr count: 55687
2021/12/25 06:06:25 Waiting for src bucket item count to get to: 200000 curr count: 55687
2021/12/25 06:06:30 Waiting for src bucket item count to get to: 200000 curr count: 72260
2021/12/25 06:06:35 Waiting for src bucket item count to get to: 200000 curr count: 72260
2021/12/25 06:06:40 Waiting for src bucket item count to get to: 200000 curr count: 87657
2021/12/25 06:06:45 Waiting for src bucket item count to get to: 200000 curr count: 87657
2021/12/25 06:06:50 Waiting for src bucket item count to get to: 200000 curr count: 87657
2021/12/25 06:06:55 Waiting for src bucket item count to get to: 200000 curr count: 104207
2021/12/25 06:07:00 Waiting for src bucket item count to get to: 200000 curr count: 104207
2021/12/25 06:07:05 Waiting for src bucket item count to get to: 200000 curr count: 120889
2021/12/25 06:07:10 Waiting for src bucket item count to get to: 200000 curr count: 120889
2021/12/25 06:07:15 Waiting for src bucket item count to get to: 200000 curr count: 137353
2021/12/25 06:07:20 Waiting for src bucket item count to get to: 200000 curr count: 137353
2021/12/25 06:07:25 Waiting for src bucket item count to get to: 200000 curr count: 154042
2021/12/25 06:07:30 Waiting for src bucket item count to get to: 200000 curr count: 154042
2021/12/25 06:07:35 Waiting for src bucket item count to get to: 200000 curr count: 170261
2021/12/25 06:07:40 Waiting for src bucket item count to get to: 200000 curr count: 170261
2021/12/25 06:07:45 Waiting for src bucket item count to get to: 200000 curr count: 185782
2021/12/25 06:07:50 Waiting for src bucket item count to get to: 200000 curr count: 185782
2021/12/25 06:07:55 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:07:55 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:08:00 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/25 06:08:05 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/25 06:08:10 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/25 06:08:15 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/25 06:08:15 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/25 06:08:15 No eventing-con process running
2021/12/25 06:08:15 Function: TestDuplicateWithKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn deleting in background"
}
2021/12/25 06:08:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:08:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:08:27 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:08:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:08:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:08:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:08:39 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:08:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebIn (223.88s)
=== RUN   TestDuplicateWithKVRebOut
2021/12/25 06:08:44 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 06:08:44 Removing node(s): [] from the cluster
2021/12/25 06:08:49 Rebalance progress: 100
2021/12/25 06:08:54 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/25 06:08:54 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/25 06:08:54 No eventing-con process running
2021/12/25 06:08:54 Function: TestDuplicateWithKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebOut not found"
}
2021/12/25 06:08:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:09:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:09:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:09:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:09:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:09:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:09:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:09:13 Deploying app: TestDuplicateWithKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:09:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut stored in temp store"
}
2021/12/25 06:09:19 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/25 06:09:24 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/25 06:09:29 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/25 06:09:34 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/25 06:09:34 App: TestDuplicateWithKVRebOut got deployed
2021/12/25 06:09:34 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:09:34 Starting bucket ops to source bucket
2021/12/25 06:09:39 Rebalance progress: 16.66666666666667
2021/12/25 06:09:44 Rebalance progress: 16.66666666666667
2021/12/25 06:09:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:09:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:09:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:09:49 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:09:49 Waiting for src bucket item count to get to: 200000 curr count: 8070
2021/12/25 06:09:54 Waiting for src bucket item count to get to: 200000 curr count: 24188
2021/12/25 06:09:59 Waiting for src bucket item count to get to: 200000 curr count: 24188
2021/12/25 06:10:04 Waiting for src bucket item count to get to: 200000 curr count: 40221
2021/12/25 06:10:09 Waiting for src bucket item count to get to: 200000 curr count: 40221
2021/12/25 06:10:14 Waiting for src bucket item count to get to: 200000 curr count: 56347
2021/12/25 06:10:19 Waiting for src bucket item count to get to: 200000 curr count: 56347
2021/12/25 06:10:24 Waiting for src bucket item count to get to: 200000 curr count: 56347
2021/12/25 06:10:29 Waiting for src bucket item count to get to: 200000 curr count: 72505
2021/12/25 06:10:34 Waiting for src bucket item count to get to: 200000 curr count: 72505
2021/12/25 06:10:39 Waiting for src bucket item count to get to: 200000 curr count: 87975
2021/12/25 06:10:44 Waiting for src bucket item count to get to: 200000 curr count: 87975
2021/12/25 06:10:49 Waiting for src bucket item count to get to: 200000 curr count: 104154
2021/12/25 06:10:54 Waiting for src bucket item count to get to: 200000 curr count: 104154
2021/12/25 06:10:59 Waiting for src bucket item count to get to: 200000 curr count: 120399
2021/12/25 06:11:04 Waiting for src bucket item count to get to: 200000 curr count: 136763
2021/12/25 06:11:09 Waiting for src bucket item count to get to: 200000 curr count: 136763
2021/12/25 06:11:14 Waiting for src bucket item count to get to: 200000 curr count: 152974
2021/12/25 06:11:19 Waiting for src bucket item count to get to: 200000 curr count: 152974
2021/12/25 06:11:24 Waiting for src bucket item count to get to: 200000 curr count: 169135
2021/12/25 06:11:29 Waiting for src bucket item count to get to: 200000 curr count: 169135
2021/12/25 06:11:34 Waiting for src bucket item count to get to: 200000 curr count: 184245
2021/12/25 06:11:39 Waiting for src bucket item count to get to: 200000 curr count: 184245
2021/12/25 06:11:44 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:11:44 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:11:49 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/25 06:11:54 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/25 06:11:59 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/25 06:12:04 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/25 06:12:04 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/25 06:12:04 No eventing-con process running
2021/12/25 06:12:04 Function: TestDuplicateWithKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut deleting in background"
}
2021/12/25 06:12:06 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:12:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:12:16 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:12:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:12:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:12:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:12:28 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:12:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebOut (228.97s)
=== RUN   TestDuplicateWithKVSwapReb
2021/12/25 06:12:38 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/25 06:12:38 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/25 06:12:38 No eventing-con process running
2021/12/25 06:12:38 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVSwapReb not found"
}
2021/12/25 06:12:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:12:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:12:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:12:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:12:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:12:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:12:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:12:56 Deploying app: TestDuplicateWithKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:12:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb stored in temp store"
}
2021/12/25 06:13:02 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/25 06:13:07 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/25 06:13:13 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/25 06:13:18 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/25 06:13:18 App: TestDuplicateWithKVSwapReb got deployed
2021/12/25 06:13:18 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 06:13:18 Starting bucket ops to source bucket
2021/12/25 06:13:18 Removing node(s): [] from the cluster
2021/12/25 06:13:23 Rebalance progress: 16.66666666666667
2021/12/25 06:13:28 Rebalance progress: 16.66666666666667
2021/12/25 06:13:33 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:13:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:13:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:13:33 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:13:33 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 06:13:34 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:13:39 Rebalance progress: 16.66666666666667
2021/12/25 06:13:44 Rebalance progress: 16.66666666666667
2021/12/25 06:13:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:13:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:13:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:13:49 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:13:49 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:13:54 Rebalance progress: 16.66666666666667
2021/12/25 06:13:59 Rebalance progress: 16.66666666666667
2021/12/25 06:14:04 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:14:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:14:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:14:04 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:14:04 Waiting for src bucket item count to get to: 200000 curr count: 66387
2021/12/25 06:14:09 Waiting for src bucket item count to get to: 200000 curr count: 66387
2021/12/25 06:14:14 Waiting for src bucket item count to get to: 200000 curr count: 81677
2021/12/25 06:14:19 Waiting for src bucket item count to get to: 200000 curr count: 81677
2021/12/25 06:14:24 Waiting for src bucket item count to get to: 200000 curr count: 98235
2021/12/25 06:14:29 Waiting for src bucket item count to get to: 200000 curr count: 98235
2021/12/25 06:14:34 Waiting for src bucket item count to get to: 200000 curr count: 114967
2021/12/25 06:14:39 Waiting for src bucket item count to get to: 200000 curr count: 114967
2021/12/25 06:14:44 Waiting for src bucket item count to get to: 200000 curr count: 131079
2021/12/25 06:14:49 Waiting for src bucket item count to get to: 200000 curr count: 131079
2021/12/25 06:14:54 Waiting for src bucket item count to get to: 200000 curr count: 147379
2021/12/25 06:14:59 Waiting for src bucket item count to get to: 200000 curr count: 147379
2021/12/25 06:15:04 Waiting for src bucket item count to get to: 200000 curr count: 163884
2021/12/25 06:15:09 Waiting for src bucket item count to get to: 200000 curr count: 163884
2021/12/25 06:15:14 Waiting for src bucket item count to get to: 200000 curr count: 179056
2021/12/25 06:15:19 Waiting for src bucket item count to get to: 200000 curr count: 179056
2021/12/25 06:15:24 Waiting for src bucket item count to get to: 200000 curr count: 195316
2021/12/25 06:15:29 Waiting for src bucket item count to get to: 200000 curr count: 195316
2021/12/25 06:15:34 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:15:34 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:15:39 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/25 06:15:44 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/25 06:15:49 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/25 06:15:54 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/25 06:15:54 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/25 06:15:54 No eventing-con process running
2021/12/25 06:15:54 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb deleting in background"
}
2021/12/25 06:15:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:16:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:16:06 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:16:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:16:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:16:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:16:18 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/25 06:16:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVSwapReb (230.11s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpOneByOne
2021/12/25 06:16:28 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:16:28 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:16:28 No eventing-con process running
2021/12/25 06:16:28 Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/25 06:16:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:16:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:16:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:16:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:16:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:16:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:16:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:16:46 Deploying app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:16:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/25 06:16:53 Starting bucket ops to source bucket
2021/12/25 06:16:58 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:17:03 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:17:08 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:17:08 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:17:08 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:17:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:17:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:17:13 Rebalance progress: 16.66666666666667
2021/12/25 06:17:18 Rebalance progress: 16.66666666666667
2021/12/25 06:17:23 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:17:23 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:17:23 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:17:23 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:17:28 Rebalance progress: 16.66666666666667
2021/12/25 06:17:33 Rebalance progress: 16.66666666666667
2021/12/25 06:17:38 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:17:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:17:38 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/25 06:17:38 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 06:17:44 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:17:49 Rebalance progress: 12.5
2021/12/25 06:17:54 Rebalance progress: 12.5
2021/12/25 06:17:59 Rebalance progress: 12.5
2021/12/25 06:18:04 Rebalance progress: 98.58870967741935
2021/12/25 06:18:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:18:09 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:18:14 Rebalance progress: 12.5
2021/12/25 06:18:19 Rebalance progress: 12.5
2021/12/25 06:18:24 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:18:24 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:18:29 Rebalance progress: 12.5
2021/12/25 06:18:34 Rebalance progress: 12.5
2021/12/25 06:18:39 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:18:39 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 06:18:44 Rebalance progress: 12.5
2021/12/25 06:18:49 Rebalance progress: 12.5
2021/12/25 06:18:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:18:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:18:59 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:19:04 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:19:09 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:19:14 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:19:14 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:19:14 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:19:14 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/25 06:19:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:19:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:19:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12140
2021/12/25 06:19:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:19:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:19:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:19:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12140
2021/12/25 06:19:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpOneByOne (199.83s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce
2021/12/25 06:19:48 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/25 06:19:48 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/25 06:19:48 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:19:48 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce not found"
}
2021/12/25 06:19:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:19:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:19:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:19:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:01 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:20:01 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:06 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:20:06 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:20:07 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce stored in temp store"
}
2021/12/25 06:20:12 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:20:17 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/25 06:20:22 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/25 06:20:27 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/25 06:20:27 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:27 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:27 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:20:27 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:20:28 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:20:28 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 06:20:28 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:20:33 Rebalance progress: 12.5
2021/12/25 06:20:38 Rebalance progress: 12.5
2021/12/25 06:20:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:20:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/25 06:20:49 Rebalance progress: 12.5
2021/12/25 06:20:54 Rebalance progress: 12.5
2021/12/25 06:20:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:20:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:21:04 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/25 06:21:09 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/25 06:21:14 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/25 06:21:19 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/25 06:21:19 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/25 06:21:19 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:21:19 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce deleting in background"
}
2021/12/25 06:21:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:21:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:21:31 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3055
2021/12/25 06:21:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3055
2021/12/25 06:21:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:21:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:21:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:21:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4596
2021/12/25 06:21:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce (129.65s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings
2021/12/25 06:21:57 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 06:21:57 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/25 06:21:57 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:21:57 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/25 06:21:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:22:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:22:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:22:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:22:16 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:22:17 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/25 06:22:22 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:22:27 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/25 06:22:32 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/25 06:22:37 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/25 06:22:37 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:22:37 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:22:37 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:22:42 Rebalance progress: 12.5
2021/12/25 06:22:47 Rebalance progress: 12.5
2021/12/25 06:22:52 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:22:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:22:52 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:22:53 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:22:58 Rebalance progress: 12.5
2021/12/25 06:23:03 Rebalance progress: 12.5
2021/12/25 06:23:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:23:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 06:23:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:23:13 Rebalance progress: 12.5
2021/12/25 06:23:18 Rebalance progress: 12.5
2021/12/25 06:23:23 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:23:23 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:23:28 Rebalance progress: 12.5
2021/12/25 06:23:33 Rebalance progress: 12.5
2021/12/25 06:23:38 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:23:39 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:23:44 Rebalance progress: 12.5
2021/12/25 06:23:49 Rebalance progress: 12.5
2021/12/25 06:23:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:23:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:23:54 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 06:23:59 Rebalance progress: 12.5
2021/12/25 06:24:04 Rebalance progress: 12.5
2021/12/25 06:24:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:24:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:24:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:24:14 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 06:24:19 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 06:24:24 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 06:24:29 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/25 06:24:29 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/25 06:24:29 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:24:29 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/25 06:24:31 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:24:36 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:24:41 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8478
2021/12/25 06:24:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8478
2021/12/25 06:24:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:24:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:24:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:24:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10651
2021/12/25 06:24:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings (190.14s)
=== RUN   TestEventingSwapRebOnUpdateBucketOp
2021/12/25 06:25:08 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/25 06:25:08 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/25 06:25:08 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:25:08 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebOnUpdateBucketOp not found"
}
2021/12/25 06:25:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:25:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:25:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:25:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:25:21 eventingSetup.go:125: Deploying app: TestEventingSwapRebOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:25:22 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp stored in temp store"
}
2021/12/25 06:25:27 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:25:32 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/25 06:25:37 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/25 06:25:42 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/25 06:25:42 clusterSetup.go:354: App: TestEventingSwapRebOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:25:42 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:25:43 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:25:48 Rebalance progress: 12.5
2021/12/25 06:25:53 Rebalance progress: 12.5
2021/12/25 06:25:58 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:25:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:25:58 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:25:58 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:26:03 Rebalance progress: 12.5
2021/12/25 06:26:08 Rebalance progress: 12.5
2021/12/25 06:26:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:26:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:26:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:26:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:26:18 Rebalance progress: 12.5
2021/12/25 06:26:23 Rebalance progress: 12.5
2021/12/25 06:26:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:26:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:26:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:26:33 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/25 06:26:38 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/25 06:26:43 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/25 06:26:48 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/25 06:26:48 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/25 06:26:48 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:26:48 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp deleting in background"
}
2021/12/25 06:26:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:26:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:27:00 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2060
2021/12/25 06:27:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2060
2021/12/25 06:27:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:27:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:27:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:27:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6099
2021/12/25 06:27:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebOnUpdateBucketOp (139.42s)
=== RUN   TestMetaRollbackWithEventingReb
2021/12/25 06:27:22 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:27:22 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:27:27 Rebalance progress: 100
2021/12/25 06:27:32 Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/25 06:27:32 App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/25 06:27:32 No eventing-con process running
2021/12/25 06:27:32 Function: TestMetaRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaRollbackWithEventingReb not found"
}
2021/12/25 06:27:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:27:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:27:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:27:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:27:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:27:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:27:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:27:46 Deploying app: TestMetaRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:27:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb stored in temp store"
}
2021/12/25 06:27:52 Starting bucket ops to source bucket
2021/12/25 06:27:57 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/25 06:28:02 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/25 06:28:07 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/25 06:28:07 App: TestMetaRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:28:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:28:07 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:28:12 Rebalance progress: 12.5
2021/12/25 06:28:17 Rebalance progress: 12.5
2021/12/25 06:28:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:28:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/25 06:28:22 bucketOps.go:134: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/25 06:28:27 Rebalance progress: 12.5
2021/12/25 06:28:32 Rebalance progress: 12.5
2021/12/25 06:28:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:28:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:28:43 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/25 06:28:48 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/25 06:28:53 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/25 06:28:58 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/25 06:28:58 clusterSetup.go:405: App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/25 06:28:58 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:28:58 eventingSetup.go:418: Function: TestMetaRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb deleting in background"
}
2021/12/25 06:29:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:29:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:29:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:29:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:29:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:29:16 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4552
2021/12/25 06:29:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaRollbackWithEventingReb (119.30s)
=== RUN   TestMetaPartialRollbackWithEventingReb
2021/12/25 06:29:21 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:29:22 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:29:27 Rebalance progress: 100
2021/12/25 06:29:32 Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/25 06:29:32 App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/25 06:29:32 No eventing-con process running
2021/12/25 06:29:32 Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaPartialRollbackWithEventingReb not found"
}
2021/12/25 06:29:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:29:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:29:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:29:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:29:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:29:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:29:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:29:45 Deploying app: TestMetaPartialRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:29:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb stored in temp store"
}
2021/12/25 06:29:51 Starting bucket ops to source bucket
2021/12/25 06:29:56 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/25 06:30:01 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/25 06:30:06 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/25 06:30:07 App: TestMetaPartialRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:30:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:30:07 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:30:12 Rebalance progress: 12.5
2021/12/25 06:30:17 Rebalance progress: 12.5
2021/12/25 06:30:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:30:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/25 06:30:22 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/25 06:30:27 Rebalance progress: 12.5
2021/12/25 06:30:32 Rebalance progress: 12.5
2021/12/25 06:30:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:30:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:30:37 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/25 06:30:42 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/25 06:30:47 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/25 06:30:52 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/25 06:30:57 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/25 06:30:57 clusterSetup.go:405: App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/25 06:30:57 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:30:57 eventingSetup.go:418: Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb deleting in background"
}
2021/12/25 06:30:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 37541
2021/12/25 06:31:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:31:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:21 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 32839
2021/12/25 06:31:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 32839
2021/12/25 06:31:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaPartialRollbackWithEventingReb (129.57s)
=== RUN   TestEventingRebMultipleHandlersOneByOne
2021/12/25 06:31:36 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/25 06:31:36 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/25 06:31:36 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:31:36 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op not found"
}
2021/12/25 06:31:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:31:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:54 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:31:56 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:31:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:31:56 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:31:57 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update stored in temp store"
}
2021/12/25 06:32:02 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:32:07 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/25 06:32:12 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/25 06:32:17 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/25 06:32:17 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got deployed
2021/12/25 06:32:22 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/25 06:32:27 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/25 06:32:32 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/25 06:32:32 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:32:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:32:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:32:32 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:32:32 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:32:37 Rebalance progress: 12.5
2021/12/25 06:32:42 Rebalance progress: 12.5
2021/12/25 06:32:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:32:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:32:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:32:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:32:47 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:32:53 Rebalance progress: 12.5
2021/12/25 06:32:58 Rebalance progress: 12.5
2021/12/25 06:33:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:33:03 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 06:33:03 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:33:09 Rebalance progress: 12.5
2021/12/25 06:33:14 Rebalance progress: 12.5
2021/12/25 06:33:19 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:33:19 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:33:24 Rebalance progress: 12.5
2021/12/25 06:33:29 Rebalance progress: 12.5
2021/12/25 06:33:34 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:33:34 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:33:39 Rebalance progress: 12.5
2021/12/25 06:33:44 Rebalance progress: 12.5
2021/12/25 06:33:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:33:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:33:49 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 06:33:54 Rebalance progress: 12.5
2021/12/25 06:33:59 Rebalance progress: 12.5
2021/12/25 06:34:04 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:34:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:34:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:34:09 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/25 06:34:14 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/25 06:34:19 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/25 06:34:24 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/25 06:34:24 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/25 06:34:24 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:34:24 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op deleting in background"
}
2021/12/25 06:34:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:34:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:34:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12210
2021/12/25 06:34:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:34:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:34:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:34:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12210
2021/12/25 06:34:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:34:54 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/25 06:34:59 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/25 06:35:04 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/25 06:35:09 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/25 06:35:10 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got un-deployed
2021/12/25 06:35:10 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:35:10 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update deleting in background"
}
2021/12/25 06:35:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:35:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersOneByOne (237.20s)
=== RUN   TestEventingRebMultipleHandlersAllAtOnce
2021/12/25 06:35:33 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/25 06:35:33 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/25 06:35:33 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:35:33 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op not found"
}
2021/12/25 06:35:35 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:40 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:35:42 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:52 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:35:53 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:35:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:35:53 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:35:54 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update stored in temp store"
}
2021/12/25 06:35:59 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:36:04 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/25 06:36:09 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/25 06:36:14 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/25 06:36:14 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got deployed
2021/12/25 06:36:19 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/25 06:36:24 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/25 06:36:29 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/25 06:36:29 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:36:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:36:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:36:29 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:36:29 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:36:30 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/25 06:36:30 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:36:35 Rebalance progress: 12.5
2021/12/25 06:36:40 Rebalance progress: 12.5
2021/12/25 06:36:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:36:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:36:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:36:45 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/25 06:36:50 Rebalance progress: 12.5
2021/12/25 06:36:55 Rebalance progress: 12.5
2021/12/25 06:37:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:37:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:37:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:37:05 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/25 06:37:10 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/25 06:37:15 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/25 06:37:20 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/25 06:37:20 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/25 06:37:20 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:37:20 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op deleting in background"
}
2021/12/25 06:37:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:37:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:37:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6122
2021/12/25 06:37:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6122
2021/12/25 06:37:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:37:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:37:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:37:51 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4459
2021/12/25 06:37:56 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4459
2021/12/25 06:37:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:38:01 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/25 06:38:06 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/25 06:38:11 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/25 06:38:16 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/25 06:38:16 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got un-deployed
2021/12/25 06:38:16 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:38:16 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update deleting in background"
}
2021/12/25 06:38:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:38:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:38:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:38:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:38:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:38:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersAllAtOnce (186.54s)
=== RUN   TestEventingSwapRebMultipleHandlers
2021/12/25 06:38:40 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/25 06:38:40 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/25 06:38:40 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:38:40 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update not found"
}
2021/12/25 06:38:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:38:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:38:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:38:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:38:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:38:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:38:58 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/25 06:38:58 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/25 06:38:58 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:38:58 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op not found"
}
2021/12/25 06:39:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:39:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:39:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:39:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:39:12 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlersbucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:39:13 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:39:13 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:39:14 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op stored in temp store"
}
2021/12/25 06:39:19 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:39:24 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/25 06:39:29 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/25 06:39:34 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/25 06:39:34 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got deployed
2021/12/25 06:39:39 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/25 06:39:44 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/25 06:39:49 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/25 06:39:49 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:39:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:39:49 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:39:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:39:55 Rebalance progress: 12.5
2021/12/25 06:40:00 Rebalance progress: 12.5
2021/12/25 06:40:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:40:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:40:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:40:10 Rebalance progress: 12.5
2021/12/25 06:40:15 Rebalance progress: 12.5
2021/12/25 06:40:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:40:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:40:25 Rebalance progress: 12.5
2021/12/25 06:40:30 Rebalance progress: 12.5
2021/12/25 06:40:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:40:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:40:41 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/25 06:40:46 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/25 06:40:51 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/25 06:40:56 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/25 06:40:56 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/25 06:40:56 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:40:56 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update deleting in background"
}
2021/12/25 06:40:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:41:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:41:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7642
2021/12/25 06:41:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:41:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:41:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:41:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3530
2021/12/25 06:41:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3530
2021/12/25 06:41:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:41:31 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/25 06:41:36 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/25 06:41:41 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/25 06:41:46 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/25 06:41:46 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/25 06:41:46 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:41:46 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op deleting in background"
}
2021/12/25 06:41:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:41:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:41:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:41:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:41:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:41:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebMultipleHandlers (209.67s)
=== RUN   TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/25 06:42:09 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:42:09 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:42:09 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:42:09 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/25 06:42:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:42:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:42:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:42:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:42:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:42:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:42:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:42:28 eventingSetup.go:125: Deploying app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:42:29 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/25 06:42:34 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:42:39 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:42:44 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:42:49 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:42:49 clusterSetup.go:354: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:42:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:42:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:42:49 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:42:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:43:10 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:43:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:43:45 Rebalance progress: 12.5
2021/12/25 06:43:50 Rebalance progress: 12.5
2021/12/25 06:43:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:43:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:43:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:43:56 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:44:01 Rebalance progress: 12.5
2021/12/25 06:44:06 Rebalance progress: 12.5
2021/12/25 06:44:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:44:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:44:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:44:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:44:16 Rebalance progress: 12.5
2021/12/25 06:44:21 Rebalance progress: 12.5
2021/12/25 06:44:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:44:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:44:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:44:31 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:44:36 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:44:41 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:44:46 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:44:46 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:44:46 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:44:46 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/25 06:44:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:44:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:44:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11186
2021/12/25 06:44:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:45:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:45:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne (185.41s)
=== RUN   TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/25 06:45:15 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:45:15 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:45:15 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:45:15 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/25 06:45:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:45:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:45:23 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:45:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:45:33 eventingSetup.go:125: Deploying app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:45:34 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/25 06:45:39 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:45:44 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:45:49 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:45:54 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/25 06:45:54 clusterSetup.go:354: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:45:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:45:54 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:45:55 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:46:15 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:46:25 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:46:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:46:45 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:46:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:46:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:46:56 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:47:16 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:27 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:27 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:47:27 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:47:27 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:47:47 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:47:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:47:57 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:47:58 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:48:18 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:48:28 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:48:33 Rebalance progress: 12.5
2021/12/25 06:48:38 Rebalance progress: 12.5
2021/12/25 06:48:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:48:43 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:48:44 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:48:49 Rebalance progress: 12.5
2021/12/25 06:48:54 Rebalance progress: 12.5
2021/12/25 06:48:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:48:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:48:59 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:49:04 Rebalance progress: 12.5
2021/12/25 06:49:09 Rebalance progress: 12.5
2021/12/25 06:49:14 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:49:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:49:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:49:19 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:49:24 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:49:29 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:49:34 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/25 06:49:34 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/25 06:49:34 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:49:34 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/25 06:49:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:49:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:49:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 21431
2021/12/25 06:49:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:49:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:49:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:49:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 21431
2021/12/25 06:49:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne (292.78s)
=== RUN   TestEventingFailoverOnUpdateBucketOp
2021/12/25 06:50:07 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/25 06:50:07 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/25 06:50:07 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:50:07 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingFailoverOnUpdateBucketOp not found"
}
2021/12/25 06:50:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:50:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:50:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:50:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:50:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:50:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:50:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:50:26 eventingSetup.go:125: Deploying app: TestEventingFailoverOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:50:27 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp stored in temp store"
}
2021/12/25 06:50:32 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:50:37 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/25 06:50:42 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/25 06:50:47 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/25 06:50:47 clusterSetup.go:354: App: TestEventingFailoverOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:50:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:50:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:50:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:50:48 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:50:53 Rebalance progress: 12.5
2021/12/25 06:50:58 Rebalance progress: 12.5
2021/12/25 06:51:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:51:03 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:51:13 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:51:18 Rebalance progress: 12.5
2021/12/25 06:51:23 Rebalance progress: 12.5
2021/12/25 06:51:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:51:28 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 06:51:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:51:34 Rebalance progress: 12.5
2021/12/25 06:51:39 Rebalance progress: 12.5
2021/12/25 06:51:44 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:51:44 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:51:49 Rebalance progress: 12.5
2021/12/25 06:51:54 Rebalance progress: 12.5
2021/12/25 06:51:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:51:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:52:04 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/25 06:52:09 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/25 06:52:14 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/25 06:52:19 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/25 06:52:19 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/25 06:52:19 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:52:19 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp deleting in background"
}
2021/12/25 06:52:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:52:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:52:31 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6384
2021/12/25 06:52:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6384
2021/12/25 06:52:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:52:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:52:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:52:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8641
2021/12/25 06:52:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingFailoverOnUpdateBucketOp (169.99s)
=== RUN   TestEventingKVRebalanceOnUpdateBucketOp
2021/12/25 06:52:57 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/25 06:52:57 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/25 06:52:57 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:52:57 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingKVRebalanceOnUpdateBucketOp not found"
}
2021/12/25 06:52:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:53:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:53:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:53:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:53:16 eventingSetup.go:125: Deploying app: TestEventingKVRebalanceOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:53:17 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp stored in temp store"
}
2021/12/25 06:53:22 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:53:27 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/25 06:53:32 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/25 06:53:37 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/25 06:53:37 clusterSetup.go:354: App: TestEventingKVRebalanceOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:53:37 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing,kv to the cluster
2021/12/25 06:53:38 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:53:43 Rebalance progress: 12.5
2021/12/25 06:53:48 Rebalance progress: 12.5
2021/12/25 06:53:53 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:53:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:53:53 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing,kv to the cluster
2021/12/25 06:53:53 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:53:58 Rebalance progress: 12.5
2021/12/25 06:54:03 Rebalance progress: 12.5
2021/12/25 06:54:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:54:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:54:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:54:08 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 06:54:13 Rebalance progress: 12.5
2021/12/25 06:54:18 Rebalance progress: 12.5
2021/12/25 06:54:23 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:54:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:54:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:54:28 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/25 06:54:33 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/25 06:54:38 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/25 06:54:43 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/25 06:54:43 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/25 06:54:43 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:54:44 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp deleting in background"
}
2021/12/25 06:54:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:54:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:54:55 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6103
2021/12/25 06:54:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:54:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:55:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:55:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6103
2021/12/25 06:55:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingKVRebalanceOnUpdateBucketOp (139.56s)
=== RUN   TestEventingRebInWithOps
2021/12/25 06:55:17 clusterSetup.go:397: Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/25 06:55:17 clusterSetup.go:405: App: TestEventingRebInWithOps got un-deployed
2021/12/25 06:55:17 eventingSetup.go:46: No eventing-con process running
2021/12/25 06:55:17 eventingSetup.go:418: Function: TestEventingRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithOps not found"
}
2021/12/25 06:55:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:55:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:55:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 06:55:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:55:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:55:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:55:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 06:55:36 eventingSetup.go:125: Deploying app: TestEventingRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 06:55:37 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps stored in temp store"
}
2021/12/25 06:55:42 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/25 06:55:47 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/25 06:55:52 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/25 06:55:57 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/25 06:55:57 clusterSetup.go:354: App: TestEventingRebInWithOps got deployed
2021/12/25 06:55:57 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 06:55:57 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 06:55:57 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 06:56:03 Rebalance progress: 12.5
2021/12/25 06:56:08 Rebalance progress: 12.5
2021/12/25 06:56:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:56:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:56:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:56:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 06:56:18 Rebalance progress: 12.5
2021/12/25 06:56:23 Rebalance progress: 12.5
2021/12/25 06:56:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:56:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 06:56:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 06:56:28 Waiting for src bucket item count to get to: 200000 curr count: 34801
2021/12/25 06:56:33 Waiting for src bucket item count to get to: 200000 curr count: 51039
2021/12/25 06:56:38 Waiting for src bucket item count to get to: 200000 curr count: 51039
2021/12/25 06:56:43 Waiting for src bucket item count to get to: 200000 curr count: 67074
2021/12/25 06:56:48 Waiting for src bucket item count to get to: 200000 curr count: 67074
2021/12/25 06:56:53 Waiting for src bucket item count to get to: 200000 curr count: 82008
2021/12/25 06:56:58 Waiting for src bucket item count to get to: 200000 curr count: 82008
2021/12/25 06:57:03 Waiting for src bucket item count to get to: 200000 curr count: 98479
2021/12/25 06:57:08 Waiting for src bucket item count to get to: 200000 curr count: 98479
2021/12/25 06:57:13 Waiting for src bucket item count to get to: 200000 curr count: 115069
2021/12/25 06:57:18 Waiting for src bucket item count to get to: 200000 curr count: 115069
2021/12/25 06:57:23 Waiting for src bucket item count to get to: 200000 curr count: 131466
2021/12/25 06:57:28 Waiting for src bucket item count to get to: 200000 curr count: 131466
2021/12/25 06:57:33 Waiting for src bucket item count to get to: 200000 curr count: 147682
2021/12/25 06:57:38 Waiting for src bucket item count to get to: 200000 curr count: 147682
2021/12/25 06:57:43 Waiting for src bucket item count to get to: 200000 curr count: 147682
2021/12/25 06:57:48 Waiting for src bucket item count to get to: 200000 curr count: 164131
2021/12/25 06:57:53 Waiting for src bucket item count to get to: 200000 curr count: 164131
2021/12/25 06:57:58 Waiting for src bucket item count to get to: 200000 curr count: 178819
2021/12/25 06:58:03 Waiting for src bucket item count to get to: 200000 curr count: 178819
2021/12/25 06:58:08 Waiting for src bucket item count to get to: 200000 curr count: 194901
2021/12/25 06:58:13 Waiting for src bucket item count to get to: 200000 curr count: 194901
2021/12/25 06:58:18 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 06:58:18 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 06:58:18 Starting bucket ops to source bucket
2021/12/25 07:02:04 Waiting for dst bucket item count to get to: 0 curr count: 17441
2021/12/25 07:02:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:02:04 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:02:09 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/25 07:02:14 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/25 07:02:19 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/25 07:02:24 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/25 07:02:24 App: TestEventingRebInWithOps got un-deployed
2021/12/25 07:02:24 No eventing-con process running
2021/12/25 07:02:24 Function: TestEventingRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps deleting in background"
}
2021/12/25 07:02:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:02:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:02:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:02:32 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:02:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:02:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithOps (450.46s)
=== RUN   TestEventingRebOutWithOps
2021/12/25 07:02:42 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 07:02:43 Removing node(s): [] from the cluster
2021/12/25 07:02:48 Rebalance progress: 100
2021/12/25 07:02:53 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/25 07:02:53 App: TestEventingRebOutWithOps got un-deployed
2021/12/25 07:02:53 No eventing-con process running
2021/12/25 07:02:53 Function: TestEventingRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithOps not found"
}
2021/12/25 07:02:55 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:03:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:03:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:03:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:03:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:03:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:03:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:03:11 Deploying app: TestEventingRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:03:12 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps stored in temp store"
}
2021/12/25 07:03:17 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/25 07:03:22 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/25 07:03:27 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/25 07:03:32 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/25 07:03:32 App: TestEventingRebOutWithOps got deployed
2021/12/25 07:03:32 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:03:32 Starting bucket ops to source bucket
2021/12/25 07:03:38 Rebalance progress: 12.5
2021/12/25 07:03:43 Rebalance progress: 12.5
2021/12/25 07:03:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:03:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:03:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:03:48 Waiting for src bucket item count to get to: 200000 curr count: 10102
2021/12/25 07:03:53 Waiting for src bucket item count to get to: 200000 curr count: 26007
2021/12/25 07:03:58 Waiting for src bucket item count to get to: 200000 curr count: 26007
2021/12/25 07:04:03 Waiting for src bucket item count to get to: 200000 curr count: 42421
2021/12/25 07:04:08 Waiting for src bucket item count to get to: 200000 curr count: 42421
2021/12/25 07:04:13 Waiting for src bucket item count to get to: 200000 curr count: 58834
2021/12/25 07:04:18 Waiting for src bucket item count to get to: 200000 curr count: 58834
2021/12/25 07:04:23 Waiting for src bucket item count to get to: 200000 curr count: 58834
2021/12/25 07:04:28 Waiting for src bucket item count to get to: 200000 curr count: 74275
2021/12/25 07:04:33 Waiting for src bucket item count to get to: 200000 curr count: 74275
2021/12/25 07:04:38 Waiting for src bucket item count to get to: 200000 curr count: 90406
2021/12/25 07:04:43 Waiting for src bucket item count to get to: 200000 curr count: 90406
2021/12/25 07:04:48 Waiting for src bucket item count to get to: 200000 curr count: 106994
2021/12/25 07:04:53 Waiting for src bucket item count to get to: 200000 curr count: 106994
2021/12/25 07:04:58 Waiting for src bucket item count to get to: 200000 curr count: 123419
2021/12/25 07:05:03 Waiting for src bucket item count to get to: 200000 curr count: 123419
2021/12/25 07:05:08 Waiting for src bucket item count to get to: 200000 curr count: 139912
2021/12/25 07:05:13 Waiting for src bucket item count to get to: 200000 curr count: 139912
2021/12/25 07:05:18 Waiting for src bucket item count to get to: 200000 curr count: 156510
2021/12/25 07:05:23 Waiting for src bucket item count to get to: 200000 curr count: 156510
2021/12/25 07:05:28 Waiting for src bucket item count to get to: 200000 curr count: 172241
2021/12/25 07:05:33 Waiting for src bucket item count to get to: 200000 curr count: 172241
2021/12/25 07:05:38 Waiting for src bucket item count to get to: 200000 curr count: 188198
2021/12/25 07:05:43 Waiting for src bucket item count to get to: 200000 curr count: 188198
2021/12/25 07:05:48 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 07:05:48 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 07:05:48 Starting bucket ops to source bucket
2021/12/25 07:09:33 Waiting for dst bucket item count to get to: 0 curr count: 16510
2021/12/25 07:09:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:09:33 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:09:38 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/25 07:09:44 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/25 07:09:49 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/25 07:09:54 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/25 07:09:54 App: TestEventingRebOutWithOps got un-deployed
2021/12/25 07:09:54 No eventing-con process running
2021/12/25 07:09:54 Function: TestEventingRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps deleting in background"
}
2021/12/25 07:09:55 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:10:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:10:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:10:02 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:10:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:10:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithOps (449.64s)
=== RUN   TestEventingSwapRebWithOps
2021/12/25 07:10:17 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/25 07:10:17 App: TestEventingSwapRebWithOps got un-deployed
2021/12/25 07:10:17 No eventing-con process running
2021/12/25 07:10:17 Function: TestEventingSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithOps not found"
}
2021/12/25 07:10:19 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:10:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:10:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:10:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:10:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:10:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:10:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:10:36 Deploying app: TestEventingSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:10:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps stored in temp store"
}
2021/12/25 07:10:42 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/25 07:10:47 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/25 07:10:52 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/25 07:10:57 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/25 07:10:57 App: TestEventingSwapRebWithOps got deployed
2021/12/25 07:10:57 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 07:10:57 Starting bucket ops to source bucket
2021/12/25 07:10:57 Removing node(s): [] from the cluster
2021/12/25 07:11:02 Rebalance progress: 12.5
2021/12/25 07:11:07 Rebalance progress: 12.5
2021/12/25 07:11:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:11:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 07:11:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:11:18 Rebalance progress: 12.5
2021/12/25 07:11:23 Rebalance progress: 12.5
2021/12/25 07:11:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:11:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 07:11:33 Rebalance progress: 12.5
2021/12/25 07:11:38 Rebalance progress: 12.5
2021/12/25 07:11:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:11:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:11:43 Waiting for src bucket item count to get to: 200000 curr count: 66075
2021/12/25 07:11:48 Waiting for src bucket item count to get to: 200000 curr count: 66075
2021/12/25 07:11:53 Waiting for src bucket item count to get to: 200000 curr count: 80881
2021/12/25 07:11:58 Waiting for src bucket item count to get to: 200000 curr count: 80881
2021/12/25 07:12:03 Waiting for src bucket item count to get to: 200000 curr count: 97329
2021/12/25 07:12:08 Waiting for src bucket item count to get to: 200000 curr count: 97329
2021/12/25 07:12:13 Waiting for src bucket item count to get to: 200000 curr count: 113714
2021/12/25 07:12:18 Waiting for src bucket item count to get to: 200000 curr count: 113714
2021/12/25 07:12:23 Waiting for src bucket item count to get to: 200000 curr count: 130154
2021/12/25 07:12:28 Waiting for src bucket item count to get to: 200000 curr count: 130154
2021/12/25 07:12:33 Waiting for src bucket item count to get to: 200000 curr count: 146565
2021/12/25 07:12:38 Waiting for src bucket item count to get to: 200000 curr count: 146565
2021/12/25 07:12:43 Waiting for src bucket item count to get to: 200000 curr count: 162748
2021/12/25 07:12:48 Waiting for src bucket item count to get to: 200000 curr count: 162748
2021/12/25 07:12:53 Waiting for src bucket item count to get to: 200000 curr count: 177913
2021/12/25 07:12:58 Waiting for src bucket item count to get to: 200000 curr count: 177913
2021/12/25 07:13:03 Waiting for src bucket item count to get to: 200000 curr count: 177913
2021/12/25 07:13:08 Waiting for src bucket item count to get to: 200000 curr count: 194135
2021/12/25 07:13:13 Waiting for src bucket item count to get to: 200000 curr count: 194135
2021/12/25 07:13:18 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 07:13:18 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 07:13:18 Starting bucket ops to source bucket
2021/12/25 07:17:04 Waiting for dst bucket item count to get to: 0 curr count: 18439
2021/12/25 07:17:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:17:04 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:17:09 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/25 07:17:14 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/25 07:17:19 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/25 07:17:24 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/25 07:17:24 App: TestEventingSwapRebWithOps got un-deployed
2021/12/25 07:17:24 No eventing-con process running
2021/12/25 07:17:25 Function: TestEventingSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps deleting in background"
}
2021/12/25 07:17:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:17:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:17:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:17:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:17:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:17:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithOps (450.93s)
=== RUN   TestKVRebInWithOps
2021/12/25 07:17:48 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/25 07:17:48 App: TestKVRebInWithOps got un-deployed
2021/12/25 07:17:48 No eventing-con process running
2021/12/25 07:17:48 Function: TestKVRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithOps not found"
}
2021/12/25 07:17:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:17:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:17:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:17:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:18:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:18:07 Deploying app: TestKVRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:18:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps stored in temp store"
}
2021/12/25 07:18:13 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/25 07:18:18 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/25 07:18:23 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/25 07:18:28 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/25 07:18:28 App: TestKVRebInWithOps got deployed
2021/12/25 07:18:28 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 07:18:28 Starting bucket ops to source bucket
2021/12/25 07:18:28 Removing node(s): [] from the cluster
2021/12/25 07:18:33 Rebalance progress: 12.5
2021/12/25 07:18:38 Rebalance progress: 12.5
2021/12/25 07:18:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:18:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:18:49 Rebalance progress: 12.5
2021/12/25 07:18:54 Rebalance progress: 12.5
2021/12/25 07:18:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:18:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:18:59 Waiting for src bucket item count to get to: 200000 curr count: 32724
2021/12/25 07:19:04 Waiting for src bucket item count to get to: 200000 curr count: 48759
2021/12/25 07:19:09 Waiting for src bucket item count to get to: 200000 curr count: 48759
2021/12/25 07:19:14 Waiting for src bucket item count to get to: 200000 curr count: 64991
2021/12/25 07:19:19 Waiting for src bucket item count to get to: 200000 curr count: 64991
2021/12/25 07:19:24 Waiting for src bucket item count to get to: 200000 curr count: 80021
2021/12/25 07:19:29 Waiting for src bucket item count to get to: 200000 curr count: 80021
2021/12/25 07:19:34 Waiting for src bucket item count to get to: 200000 curr count: 95837
2021/12/25 07:19:39 Waiting for src bucket item count to get to: 200000 curr count: 95837
2021/12/25 07:19:44 Waiting for src bucket item count to get to: 200000 curr count: 112123
2021/12/25 07:19:49 Waiting for src bucket item count to get to: 200000 curr count: 112123
2021/12/25 07:19:54 Waiting for src bucket item count to get to: 200000 curr count: 128431
2021/12/25 07:19:59 Waiting for src bucket item count to get to: 200000 curr count: 128431
2021/12/25 07:20:04 Waiting for src bucket item count to get to: 200000 curr count: 128431
2021/12/25 07:20:09 Waiting for src bucket item count to get to: 200000 curr count: 144531
2021/12/25 07:20:14 Waiting for src bucket item count to get to: 200000 curr count: 144531
2021/12/25 07:20:19 Waiting for src bucket item count to get to: 200000 curr count: 160612
2021/12/25 07:20:24 Waiting for src bucket item count to get to: 200000 curr count: 160612
2021/12/25 07:20:29 Waiting for src bucket item count to get to: 200000 curr count: 175759
2021/12/25 07:20:34 Waiting for src bucket item count to get to: 200000 curr count: 175759
2021/12/25 07:20:39 Waiting for src bucket item count to get to: 200000 curr count: 191604
2021/12/25 07:20:44 Waiting for src bucket item count to get to: 200000 curr count: 191604
2021/12/25 07:20:49 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 07:20:49 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 07:20:49 Starting bucket ops to source bucket
2021/12/25 07:24:35 Waiting for dst bucket item count to get to: 0 curr count: 19531
2021/12/25 07:24:40 Waiting for dst bucket item count to get to: 0 curr count: 974
2021/12/25 07:24:45 Waiting for dst bucket item count to get to: 0 curr count: 974
2021/12/25 07:24:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:24:45 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:24:50 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/25 07:24:55 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/25 07:25:00 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/25 07:25:05 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/25 07:25:05 App: TestKVRebInWithOps got un-deployed
2021/12/25 07:25:05 No eventing-con process running
2021/12/25 07:25:05 Function: TestKVRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps deleting in background"
}
2021/12/25 07:25:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:25:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:25:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:25:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:25:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:25:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithOps (460.61s)
=== RUN   TestKVRebOutWithOps
2021/12/25 07:25:24 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 07:25:25 Removing node(s): [] from the cluster
2021/12/25 07:25:30 Rebalance progress: 100
2021/12/25 07:25:35 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/25 07:25:35 App: TestKVRebOutWithOps got un-deployed
2021/12/25 07:25:35 No eventing-con process running
2021/12/25 07:25:35 Function: TestKVRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithOps not found"
}
2021/12/25 07:25:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:25:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:25:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:25:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:25:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:25:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:25:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:25:53 Deploying app: TestKVRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:25:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps stored in temp store"
}
2021/12/25 07:25:59 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/25 07:26:04 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/25 07:26:09 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/25 07:26:14 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/25 07:26:14 App: TestKVRebOutWithOps got deployed
2021/12/25 07:26:14 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:26:14 Starting bucket ops to source bucket
2021/12/25 07:26:19 Rebalance progress: 12.5
2021/12/25 07:26:24 Rebalance progress: 12.5
2021/12/25 07:26:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:26:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:26:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:26:29 Waiting for src bucket item count to get to: 200000 curr count: 7154
2021/12/25 07:26:34 Waiting for src bucket item count to get to: 200000 curr count: 23317
2021/12/25 07:26:39 Waiting for src bucket item count to get to: 200000 curr count: 23317
2021/12/25 07:26:44 Waiting for src bucket item count to get to: 200000 curr count: 39705
2021/12/25 07:26:49 Waiting for src bucket item count to get to: 200000 curr count: 39705
2021/12/25 07:26:54 Waiting for src bucket item count to get to: 200000 curr count: 55933
2021/12/25 07:26:59 Waiting for src bucket item count to get to: 200000 curr count: 55933
2021/12/25 07:27:05 Waiting for src bucket item count to get to: 200000 curr count: 55933
2021/12/25 07:27:10 Waiting for src bucket item count to get to: 200000 curr count: 72465
2021/12/25 07:27:15 Waiting for src bucket item count to get to: 200000 curr count: 87605
2021/12/25 07:27:20 Waiting for src bucket item count to get to: 200000 curr count: 87605
2021/12/25 07:27:25 Waiting for src bucket item count to get to: 200000 curr count: 104041
2021/12/25 07:27:30 Waiting for src bucket item count to get to: 200000 curr count: 104041
2021/12/25 07:27:35 Waiting for src bucket item count to get to: 200000 curr count: 120483
2021/12/25 07:27:40 Waiting for src bucket item count to get to: 200000 curr count: 120483
2021/12/25 07:27:45 Waiting for src bucket item count to get to: 200000 curr count: 136943
2021/12/25 07:27:50 Waiting for src bucket item count to get to: 200000 curr count: 136943
2021/12/25 07:27:55 Waiting for src bucket item count to get to: 200000 curr count: 153375
2021/12/25 07:28:00 Waiting for src bucket item count to get to: 200000 curr count: 153375
2021/12/25 07:28:05 Waiting for src bucket item count to get to: 200000 curr count: 169748
2021/12/25 07:28:10 Waiting for src bucket item count to get to: 200000 curr count: 169748
2021/12/25 07:28:15 Waiting for src bucket item count to get to: 200000 curr count: 185025
2021/12/25 07:28:20 Waiting for src bucket item count to get to: 200000 curr count: 185025
2021/12/25 07:28:25 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 07:28:25 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 07:28:25 Starting bucket ops to source bucket
2021/12/25 07:32:11 Waiting for dst bucket item count to get to: 0 curr count: 11201
2021/12/25 07:32:16 Waiting for dst bucket item count to get to: 0 curr count: 11201
2021/12/25 07:32:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:32:16 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:32:21 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/25 07:32:26 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/25 07:32:31 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/25 07:32:36 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/25 07:32:36 App: TestKVRebOutWithOps got un-deployed
2021/12/25 07:32:36 No eventing-con process running
2021/12/25 07:32:36 Function: TestKVRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps deleting in background"
}
2021/12/25 07:32:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:32:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:32:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:32:45 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:32:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:32:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithOps (450.89s)
=== RUN   TestKVSwapRebWithOps
2021/12/25 07:33:00 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/25 07:33:00 App: TestKVSwapRebWithOps got un-deployed
2021/12/25 07:33:00 No eventing-con process running
2021/12/25 07:33:00 Function: TestKVSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithOps not found"
}
2021/12/25 07:33:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:33:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:33:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:33:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:33:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:33:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:33:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:33:18 Deploying app: TestKVSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:33:19 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps stored in temp store"
}
2021/12/25 07:33:24 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/25 07:33:29 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/25 07:33:34 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/25 07:33:39 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/25 07:33:39 App: TestKVSwapRebWithOps got deployed
2021/12/25 07:33:39 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 07:33:39 Starting bucket ops to source bucket
2021/12/25 07:33:40 Removing node(s): [] from the cluster
2021/12/25 07:33:45 Rebalance progress: 12.5
2021/12/25 07:33:50 Rebalance progress: 12.5
2021/12/25 07:33:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:33:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:33:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:33:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 07:33:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:34:00 Rebalance progress: 12.5
2021/12/25 07:34:05 Rebalance progress: 12.5
2021/12/25 07:34:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:34:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:34:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:34:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 07:34:15 Rebalance progress: 12.5
2021/12/25 07:34:20 Rebalance progress: 12.5
2021/12/25 07:34:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:34:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:34:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:34:25 Waiting for src bucket item count to get to: 200000 curr count: 61352
2021/12/25 07:34:30 Waiting for src bucket item count to get to: 200000 curr count: 61352
2021/12/25 07:34:35 Waiting for src bucket item count to get to: 200000 curr count: 76537
2021/12/25 07:34:40 Waiting for src bucket item count to get to: 200000 curr count: 92623
2021/12/25 07:34:46 Waiting for src bucket item count to get to: 200000 curr count: 92623
2021/12/25 07:34:51 Waiting for src bucket item count to get to: 200000 curr count: 109041
2021/12/25 07:34:56 Waiting for src bucket item count to get to: 200000 curr count: 109041
2021/12/25 07:35:01 Waiting for src bucket item count to get to: 200000 curr count: 125113
2021/12/25 07:35:06 Waiting for src bucket item count to get to: 200000 curr count: 125113
2021/12/25 07:35:11 Waiting for src bucket item count to get to: 200000 curr count: 141552
2021/12/25 07:35:16 Waiting for src bucket item count to get to: 200000 curr count: 141552
2021/12/25 07:35:21 Waiting for src bucket item count to get to: 200000 curr count: 158119
2021/12/25 07:35:26 Waiting for src bucket item count to get to: 200000 curr count: 158119
2021/12/25 07:35:31 Waiting for src bucket item count to get to: 200000 curr count: 173304
2021/12/25 07:35:36 Waiting for src bucket item count to get to: 200000 curr count: 173304
2021/12/25 07:35:41 Waiting for src bucket item count to get to: 200000 curr count: 189102
2021/12/25 07:35:46 Waiting for src bucket item count to get to: 200000 curr count: 189102
2021/12/25 07:35:51 Waiting for src bucket item count to get to: 200000 curr count: 189102
2021/12/25 07:35:56 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/25 07:35:56 Post SET ops. Expected item count: 200000 got 200000
2021/12/25 07:35:56 Starting bucket ops to source bucket
2021/12/25 07:39:42 Waiting for dst bucket item count to get to: 0 curr count: 13774
2021/12/25 07:39:47 Waiting for dst bucket item count to get to: 0 curr count: 13774
2021/12/25 07:39:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:39:47 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:39:52 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/25 07:39:57 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/25 07:40:02 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/25 07:40:07 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/25 07:40:07 App: TestKVSwapRebWithOps got un-deployed
2021/12/25 07:40:07 No eventing-con process running
2021/12/25 07:40:07 Function: TestKVSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps deleting in background"
}
2021/12/25 07:40:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:40:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:40:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:40:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:40:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:40:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithOps (451.13s)
=== RUN   TestEventingRebInWithTimers
2021/12/25 07:40:31 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/25 07:40:31 App: TestEventingRebInWithTimers got un-deployed
2021/12/25 07:40:31 No eventing-con process running
2021/12/25 07:40:31 Function: TestEventingRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithTimers not found"
}
2021/12/25 07:40:32 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:40:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:40:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:40:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:40:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:40:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:40:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:40:49 Deploying app: TestEventingRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:40:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers stored in temp store"
}
2021/12/25 07:40:55 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/25 07:41:00 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/25 07:41:05 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/25 07:41:10 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/25 07:41:10 App: TestEventingRebInWithTimers got deployed
2021/12/25 07:41:10 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 07:41:10 Starting bucket ops to source bucket
2021/12/25 07:41:11 Removing node(s): [] from the cluster
2021/12/25 07:41:16 Rebalance progress: 12.5
2021/12/25 07:41:21 Rebalance progress: 12.5
2021/12/25 07:41:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:41:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:41:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:41:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:41:31 Rebalance progress: 12.5
2021/12/25 07:41:36 Rebalance progress: 12.5
2021/12/25 07:41:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:41:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:41:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:41:41 Waiting for src bucket item count to get to: 50000 curr count: 33819
2021/12/25 07:41:46 Waiting for src bucket item count to get to: 50000 curr count: 33819
2021/12/25 07:41:51 Waiting for src bucket item count to get to: 50000 curr count: 45368
2021/12/25 07:41:56 Waiting for src bucket item count to get to: 50000 curr count: 45368
2021/12/25 07:42:06 Waiting for dst bucket item count to get to: 50000 curr count: 43205
2021/12/25 07:42:11 Waiting for dst bucket item count to get to: 50000 curr count: 43205
2021/12/25 07:42:11 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 07:42:11 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 07:42:11 Starting bucket ops to source bucket
2021/12/25 07:43:21 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:43:26 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:43:31 Waiting for dst bucket item count to get to: 0 curr count: 46820
2021/12/25 07:43:36 Waiting for dst bucket item count to get to: 0 curr count: 46820
2021/12/25 07:43:41 Waiting for dst bucket item count to get to: 0 curr count: 36265
2021/12/25 07:43:46 Waiting for dst bucket item count to get to: 0 curr count: 36265
2021/12/25 07:43:51 Waiting for dst bucket item count to get to: 0 curr count: 15208
2021/12/25 07:43:56 Waiting for dst bucket item count to get to: 0 curr count: 15208
2021/12/25 07:44:01 Waiting for dst bucket item count to get to: 0 curr count: 1748
2021/12/25 07:44:06 Waiting for dst bucket item count to get to: 0 curr count: 1748
2021/12/25 07:44:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:44:06 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:44:11 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/25 07:44:16 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/25 07:44:21 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/25 07:44:21 App: TestEventingRebInWithTimers got un-deployed
2021/12/25 07:44:21 No eventing-con process running
2021/12/25 07:44:21 Function: TestEventingRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers deleting in background"
}
2021/12/25 07:44:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:44:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:44:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:44:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:44:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:44:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithTimers (253.73s)
=== RUN   TestEventingRebOutWithTimers
2021/12/25 07:44:39 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 07:44:40 Removing node(s): [] from the cluster
2021/12/25 07:44:45 Rebalance progress: 100
2021/12/25 07:44:50 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/25 07:44:50 App: TestEventingRebOutWithTimers got un-deployed
2021/12/25 07:44:50 No eventing-con process running
2021/12/25 07:44:50 Function: TestEventingRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithTimers not found"
}
2021/12/25 07:44:52 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:44:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:44:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:44:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:45:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:45:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:45:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:45:09 Deploying app: TestEventingRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:45:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers stored in temp store"
}
2021/12/25 07:45:15 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/25 07:45:20 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/25 07:45:25 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/25 07:45:30 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/25 07:45:30 App: TestEventingRebOutWithTimers got deployed
2021/12/25 07:45:30 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:45:30 Starting bucket ops to source bucket
2021/12/25 07:45:35 Rebalance progress: 12.5
2021/12/25 07:45:40 Rebalance progress: 12.5
2021/12/25 07:45:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:45:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:45:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:45:45 Waiting for src bucket item count to get to: 50000 curr count: 12740
2021/12/25 07:45:50 Waiting for src bucket item count to get to: 50000 curr count: 24511
2021/12/25 07:45:55 Waiting for src bucket item count to get to: 50000 curr count: 24511
2021/12/25 07:46:00 Waiting for src bucket item count to get to: 50000 curr count: 34694
2021/12/25 07:46:05 Waiting for src bucket item count to get to: 50000 curr count: 34694
2021/12/25 07:46:10 Waiting for src bucket item count to get to: 50000 curr count: 47065
2021/12/25 07:46:15 Waiting for src bucket item count to get to: 50000 curr count: 47065
2021/12/25 07:46:20 Waiting for src bucket item count to get to: 50000 curr count: 47065
2021/12/25 07:46:30 Waiting for dst bucket item count to get to: 50000 curr count: 46125
2021/12/25 07:46:35 Waiting for dst bucket item count to get to: 50000 curr count: 46125
2021/12/25 07:46:35 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 07:46:35 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 07:46:35 Starting bucket ops to source bucket
2021/12/25 07:47:45 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:47:50 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:47:55 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:48:00 Waiting for dst bucket item count to get to: 0 curr count: 43640
2021/12/25 07:48:05 Waiting for dst bucket item count to get to: 0 curr count: 43640
2021/12/25 07:48:10 Waiting for dst bucket item count to get to: 0 curr count: 23136
2021/12/25 07:48:15 Waiting for dst bucket item count to get to: 0 curr count: 23136
2021/12/25 07:48:20 Waiting for dst bucket item count to get to: 0 curr count: 10952
2021/12/25 07:48:25 Waiting for dst bucket item count to get to: 0 curr count: 10952
2021/12/25 07:48:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:48:25 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:48:30 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/25 07:48:35 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/25 07:48:40 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/25 07:48:45 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/25 07:48:45 App: TestEventingRebOutWithTimers got un-deployed
2021/12/25 07:48:45 No eventing-con process running
2021/12/25 07:48:45 Function: TestEventingRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers deleting in background"
}
2021/12/25 07:48:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:48:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:48:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:48:54 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:48:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:48:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithTimers (264.35s)
=== RUN   TestEventingSwapRebWithTimers
2021/12/25 07:49:09 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/25 07:49:09 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/25 07:49:09 No eventing-con process running
2021/12/25 07:49:09 Function: TestEventingSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithTimers not found"
}
2021/12/25 07:49:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:49:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:49:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:49:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:49:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:49:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:49:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:49:27 Deploying app: TestEventingSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:49:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers stored in temp store"
}
2021/12/25 07:49:33 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/25 07:49:38 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/25 07:49:43 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/25 07:49:48 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/25 07:49:49 App: TestEventingSwapRebWithTimers got deployed
2021/12/25 07:49:49 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 07:49:49 Starting bucket ops to source bucket
2021/12/25 07:49:50 Removing node(s): [] from the cluster
2021/12/25 07:49:55 Rebalance progress: 12.5
2021/12/25 07:50:00 Rebalance progress: 12.5
2021/12/25 07:50:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:50:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 07:50:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:50:10 Rebalance progress: 12.5
2021/12/25 07:50:15 Rebalance progress: 12.5
2021/12/25 07:50:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:50:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 07:50:26 Rebalance progress: 12.5
2021/12/25 07:50:31 Rebalance progress: 12.5
2021/12/25 07:50:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:50:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:50:36 Waiting for src bucket item count to get to: 50000 curr count: 47287
2021/12/25 07:50:41 Waiting for src bucket item count to get to: 50000 curr count: 47287
2021/12/25 07:50:51 Waiting for dst bucket item count to get to: 50000 curr count: 47923
2021/12/25 07:50:56 Waiting for dst bucket item count to get to: 50000 curr count: 47923
2021/12/25 07:50:56 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 07:50:56 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 07:50:56 Starting bucket ops to source bucket
2021/12/25 07:52:05 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:52:10 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:52:15 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:52:20 Waiting for dst bucket item count to get to: 0 curr count: 39394
2021/12/25 07:52:25 Waiting for dst bucket item count to get to: 0 curr count: 39394
2021/12/25 07:52:30 Waiting for dst bucket item count to get to: 0 curr count: 25157
2021/12/25 07:52:35 Waiting for dst bucket item count to get to: 0 curr count: 25157
2021/12/25 07:52:40 Waiting for dst bucket item count to get to: 0 curr count: 10649
2021/12/25 07:52:45 Waiting for dst bucket item count to get to: 0 curr count: 10649
2021/12/25 07:52:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:52:45 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:52:50 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/25 07:52:55 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/25 07:53:00 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/25 07:53:05 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/25 07:53:05 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/25 07:53:05 No eventing-con process running
2021/12/25 07:53:05 Function: TestEventingSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers deleting in background"
}
2021/12/25 07:53:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:53:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:53:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:53:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:53:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:53:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithTimers (260.12s)
=== RUN   TestKVRebInWithTimers
2021/12/25 07:53:29 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/25 07:53:29 App: TestKVRebInWithTimers got un-deployed
2021/12/25 07:53:29 No eventing-con process running
2021/12/25 07:53:29 Function: TestKVRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithTimers not found"
}
2021/12/25 07:53:31 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:53:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:53:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:53:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:53:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:53:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:53:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:53:47 Deploying app: TestKVRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:53:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers stored in temp store"
}
2021/12/25 07:53:53 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/25 07:53:59 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/25 07:54:04 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/25 07:54:09 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/25 07:54:09 App: TestKVRebInWithTimers got deployed
2021/12/25 07:54:09 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 07:54:09 Starting bucket ops to source bucket
2021/12/25 07:54:09 Removing node(s): [] from the cluster
2021/12/25 07:54:14 Rebalance progress: 12.5
2021/12/25 07:54:19 Rebalance progress: 12.5
2021/12/25 07:54:24 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:54:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:54:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:54:24 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:54:29 Rebalance progress: 12.5
2021/12/25 07:54:34 Rebalance progress: 12.5
2021/12/25 07:54:39 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:54:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:54:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:54:39 Waiting for src bucket item count to get to: 50000 curr count: 25891
2021/12/25 07:54:44 Waiting for src bucket item count to get to: 50000 curr count: 35430
2021/12/25 07:54:49 Waiting for src bucket item count to get to: 50000 curr count: 35430
2021/12/25 07:54:54 Waiting for src bucket item count to get to: 50000 curr count: 47983
2021/12/25 07:54:59 Waiting for src bucket item count to get to: 50000 curr count: 47983
2021/12/25 07:55:09 Waiting for dst bucket item count to get to: 50000 curr count: 48285
2021/12/25 07:55:14 Waiting for dst bucket item count to get to: 50000 curr count: 48285
2021/12/25 07:55:14 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 07:55:14 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 07:55:14 Starting bucket ops to source bucket
2021/12/25 07:56:24 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:56:29 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:56:34 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 07:56:39 Waiting for dst bucket item count to get to: 0 curr count: 41228
2021/12/25 07:56:44 Waiting for dst bucket item count to get to: 0 curr count: 41228
2021/12/25 07:56:49 Waiting for dst bucket item count to get to: 0 curr count: 25482
2021/12/25 07:56:54 Waiting for dst bucket item count to get to: 0 curr count: 25482
2021/12/25 07:56:59 Waiting for dst bucket item count to get to: 0 curr count: 11668
2021/12/25 07:57:04 Waiting for dst bucket item count to get to: 0 curr count: 11668
2021/12/25 07:57:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:57:04 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 07:57:09 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/25 07:57:15 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/25 07:57:20 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/25 07:57:25 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/25 07:57:25 App: TestKVRebInWithTimers got un-deployed
2021/12/25 07:57:25 No eventing-con process running
2021/12/25 07:57:25 Function: TestKVRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers deleting in background"
}
2021/12/25 07:57:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:57:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:57:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:57:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:57:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:57:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithTimers (259.36s)
=== RUN   TestKVRebOutWithTimers
2021/12/25 07:57:43 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 07:57:43 Removing node(s): [] from the cluster
2021/12/25 07:57:49 Rebalance progress: 100
2021/12/25 07:57:54 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/25 07:57:54 App: TestKVRebOutWithTimers got un-deployed
2021/12/25 07:57:54 No eventing-con process running
2021/12/25 07:57:54 Function: TestKVRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithTimers not found"
}
2021/12/25 07:57:55 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:58:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:58:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 07:58:02 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:58:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:58:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:58:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 07:58:12 Deploying app: TestKVRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 07:58:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers stored in temp store"
}
2021/12/25 07:58:18 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/25 07:58:23 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/25 07:58:28 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/25 07:58:33 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/25 07:58:33 App: TestKVRebOutWithTimers got deployed
2021/12/25 07:58:33 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 07:58:33 Starting bucket ops to source bucket
2021/12/25 07:58:38 Rebalance progress: 12.5
2021/12/25 07:58:43 Rebalance progress: 12.5
2021/12/25 07:58:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:58:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 07:58:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 07:58:48 Waiting for src bucket item count to get to: 50000 curr count: 7638
2021/12/25 07:58:53 Waiting for src bucket item count to get to: 50000 curr count: 20211
2021/12/25 07:58:58 Waiting for src bucket item count to get to: 50000 curr count: 20211
2021/12/25 07:59:03 Waiting for src bucket item count to get to: 50000 curr count: 31670
2021/12/25 07:59:08 Waiting for src bucket item count to get to: 50000 curr count: 31670
2021/12/25 07:59:13 Waiting for src bucket item count to get to: 50000 curr count: 42634
2021/12/25 07:59:18 Waiting for src bucket item count to get to: 50000 curr count: 42634
2021/12/25 07:59:23 Waiting for src bucket item count to get to: 50000 curr count: 42634
2021/12/25 07:59:33 Waiting for dst bucket item count to get to: 50000 curr count: 39452
2021/12/25 07:59:39 Waiting for dst bucket item count to get to: 50000 curr count: 39452
2021/12/25 07:59:44 Waiting for dst bucket item count to get to: 50000 curr count: 49178
2021/12/25 07:59:49 Waiting for dst bucket item count to get to: 50000 curr count: 49178
2021/12/25 07:59:49 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 07:59:49 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 07:59:49 Starting bucket ops to source bucket
2021/12/25 08:00:58 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:01:03 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:01:08 Waiting for dst bucket item count to get to: 0 curr count: 48430
2021/12/25 08:01:13 Waiting for dst bucket item count to get to: 0 curr count: 48430
2021/12/25 08:01:18 Waiting for dst bucket item count to get to: 0 curr count: 29772
2021/12/25 08:01:23 Waiting for dst bucket item count to get to: 0 curr count: 29772
2021/12/25 08:01:28 Waiting for dst bucket item count to get to: 0 curr count: 18213
2021/12/25 08:01:33 Waiting for dst bucket item count to get to: 0 curr count: 18213
2021/12/25 08:01:38 Waiting for dst bucket item count to get to: 0 curr count: 958
2021/12/25 08:01:43 Waiting for dst bucket item count to get to: 0 curr count: 958
2021/12/25 08:01:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:01:43 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 08:01:48 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/25 08:01:54 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/25 08:01:59 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/25 08:02:04 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/25 08:02:04 App: TestKVRebOutWithTimers got un-deployed
2021/12/25 08:02:04 No eventing-con process running
2021/12/25 08:02:04 Function: TestKVRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers deleting in background"
}
2021/12/25 08:02:05 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:02:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:02:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:02:12 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:02:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:02:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithTimers (278.91s)
=== RUN   TestKVSwapRebWithTimers
2021/12/25 08:02:27 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:02:27 App: TestKVSwapRebWithTimers got un-deployed
2021/12/25 08:02:27 No eventing-con process running
2021/12/25 08:02:27 Function: TestKVSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithTimers not found"
}
2021/12/25 08:02:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:02:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:02:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:02:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:02:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:02:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:02:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:02:46 Deploying app: TestKVSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:02:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers stored in temp store"
}
2021/12/25 08:02:52 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/25 08:02:57 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/25 08:03:02 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/25 08:03:07 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/25 08:03:07 App: TestKVSwapRebWithTimers got deployed
2021/12/25 08:03:07 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:03:07 Starting bucket ops to source bucket
2021/12/25 08:03:07 Removing node(s): [] from the cluster
2021/12/25 08:03:12 Rebalance progress: 12.5
2021/12/25 08:03:17 Rebalance progress: 12.5
2021/12/25 08:03:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:03:22 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:03:23 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:03:28 Rebalance progress: 12.5
2021/12/25 08:03:33 Rebalance progress: 12.5
2021/12/25 08:03:38 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:03:38 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:03:43 Rebalance progress: 12.5
2021/12/25 08:03:48 Rebalance progress: 12.5
2021/12/25 08:03:53 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:03:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:03:53 Waiting for src bucket item count to get to: 50000 curr count: 48394
2021/12/25 08:03:58 Waiting for src bucket item count to get to: 50000 curr count: 48394
2021/12/25 08:04:08 Waiting for dst bucket item count to get to: 50000 curr count: 47543
2021/12/25 08:04:13 Waiting for dst bucket item count to get to: 50000 curr count: 47543
2021/12/25 08:04:13 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:04:13 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:04:13 Starting bucket ops to source bucket
2021/12/25 08:05:23 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:05:28 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:05:33 Waiting for dst bucket item count to get to: 0 curr count: 49135
2021/12/25 08:05:38 Waiting for dst bucket item count to get to: 0 curr count: 49135
2021/12/25 08:05:43 Waiting for dst bucket item count to get to: 0 curr count: 37387
2021/12/25 08:05:48 Waiting for dst bucket item count to get to: 0 curr count: 37387
2021/12/25 08:05:53 Waiting for dst bucket item count to get to: 0 curr count: 25564
2021/12/25 08:05:58 Waiting for dst bucket item count to get to: 0 curr count: 25564
2021/12/25 08:06:03 Waiting for dst bucket item count to get to: 0 curr count: 4372
2021/12/25 08:06:08 Waiting for dst bucket item count to get to: 0 curr count: 4372
2021/12/25 08:06:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:06:08 Post DELETE ops. Expected item count: 0 got 0
2021/12/25 08:06:13 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:06:18 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:06:24 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:06:29 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:06:34 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/25 08:06:34 App: TestKVSwapRebWithTimers got un-deployed
2021/12/25 08:06:34 No eventing-con process running
2021/12/25 08:06:34 Function: TestKVSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers deleting in background"
}
2021/12/25 08:06:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:06:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:06:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:06:42 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:06:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:06:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithTimers (270.11s)
=== RUN   TestDuplicateTimerCreationEventingRebIn
2021/12/25 08:06:57 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:06:57 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/25 08:06:57 No eventing-con process running
2021/12/25 08:06:57 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebIn not found"
}
2021/12/25 08:06:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:07:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:07:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:07:06 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:07:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:07:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:07:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:07:16 Deploying app: TestDuplicateTimerCreationEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:07:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn stored in temp store"
}
2021/12/25 08:07:22 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/25 08:07:27 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/25 08:07:32 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/25 08:07:37 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/25 08:07:37 App: TestDuplicateTimerCreationEventingRebIn got deployed
2021/12/25 08:07:37 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:07:37 Starting bucket ops to source bucket
2021/12/25 08:07:37 Removing node(s): [] from the cluster
2021/12/25 08:07:42 Rebalance progress: 12.5
2021/12/25 08:07:47 Rebalance progress: 12.5
2021/12/25 08:07:52 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:07:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:07:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:07:52 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:07:58 Rebalance progress: 12.5
2021/12/25 08:08:03 Rebalance progress: 12.5
2021/12/25 08:08:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:08:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:08:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:08:08 Waiting for src bucket item count to get to: 50000 curr count: 28955
2021/12/25 08:08:13 Waiting for src bucket item count to get to: 50000 curr count: 40127
2021/12/25 08:08:18 Waiting for src bucket item count to get to: 50000 curr count: 40127
2021/12/25 08:08:28 Waiting for dst bucket item count to get to: 50000 curr count: 24509
2021/12/25 08:08:33 Waiting for dst bucket item count to get to: 50000 curr count: 24509
2021/12/25 08:08:38 Waiting for dst bucket item count to get to: 50000 curr count: 35977
2021/12/25 08:08:43 Waiting for dst bucket item count to get to: 50000 curr count: 35977
2021/12/25 08:08:48 Waiting for dst bucket item count to get to: 50000 curr count: 43367
2021/12/25 08:08:53 Waiting for dst bucket item count to get to: 50000 curr count: 43367
2021/12/25 08:08:53 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:08:53 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:08:58 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:09:03 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:09:08 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:09:13 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:09:18 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/25 08:09:18 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/25 08:09:18 No eventing-con process running
2021/12/25 08:09:18 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn deleting in background"
}
2021/12/25 08:09:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:09:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:09:30 Waiting for dst bucket item count to get to: 0 curr count: 4761
2021/12/25 08:09:35 Waiting for dst bucket item count to get to: 0 curr count: 4761
2021/12/25 08:09:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:09:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:09:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:09:47 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:09:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebIn (179.49s)
=== RUN   TestDuplicateTimerCreationEventingRebOut
2021/12/25 08:09:52 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:09:52 Removing node(s): [] from the cluster
2021/12/25 08:09:57 Rebalance progress: 100
2021/12/25 08:10:02 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:10:02 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/25 08:10:02 No eventing-con process running
2021/12/25 08:10:02 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebOut not found"
}
2021/12/25 08:10:04 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:10:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:10:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:10:11 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:10:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:10:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:10:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:10:21 Deploying app: TestDuplicateTimerCreationEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:10:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut stored in temp store"
}
2021/12/25 08:10:27 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/25 08:10:32 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/25 08:10:37 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/25 08:10:42 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/25 08:10:42 App: TestDuplicateTimerCreationEventingRebOut got deployed
2021/12/25 08:10:42 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:10:42 Starting bucket ops to source bucket
2021/12/25 08:10:47 Rebalance progress: 12.5
2021/12/25 08:10:52 Rebalance progress: 12.5
2021/12/25 08:10:57 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:10:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:10:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:10:57 Waiting for src bucket item count to get to: 50000 curr count: 9554
2021/12/25 08:11:02 Waiting for src bucket item count to get to: 50000 curr count: 23574
2021/12/25 08:11:07 Waiting for src bucket item count to get to: 50000 curr count: 23574
2021/12/25 08:11:12 Waiting for src bucket item count to get to: 50000 curr count: 35510
2021/12/25 08:11:17 Waiting for src bucket item count to get to: 50000 curr count: 35510
2021/12/25 08:11:22 Waiting for src bucket item count to get to: 50000 curr count: 44883
2021/12/25 08:11:27 Waiting for src bucket item count to get to: 50000 curr count: 44883
2021/12/25 08:11:37 Waiting for dst bucket item count to get to: 50000 curr count: 34477
2021/12/25 08:11:42 Waiting for dst bucket item count to get to: 50000 curr count: 34477
2021/12/25 08:11:47 Waiting for dst bucket item count to get to: 50000 curr count: 34477
2021/12/25 08:11:52 Waiting for dst bucket item count to get to: 50000 curr count: 48925
2021/12/25 08:11:57 Waiting for dst bucket item count to get to: 50000 curr count: 48925
2021/12/25 08:11:57 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:11:57 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:12:02 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:12:07 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:12:12 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:12:18 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:12:23 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/25 08:12:23 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/25 08:12:23 No eventing-con process running
2021/12/25 08:12:23 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut deleting in background"
}
2021/12/25 08:12:24 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:12:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:12:34 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:12:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:12:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:12:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:12:46 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:12:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebOut (179.53s)
=== RUN   TestDuplicateTimerCreationEventingSwapReb
2021/12/25 08:12:56 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:12:56 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/25 08:12:56 No eventing-con process running
2021/12/25 08:12:56 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingSwapReb not found"
}
2021/12/25 08:12:58 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:13:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:13:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:13:05 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:13:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:13:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:13:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:13:15 Deploying app: TestDuplicateTimerCreationEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:13:16 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb stored in temp store"
}
2021/12/25 08:13:21 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/25 08:13:26 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/25 08:13:31 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/25 08:13:36 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/25 08:13:36 App: TestDuplicateTimerCreationEventingSwapReb got deployed
2021/12/25 08:13:36 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:13:36 Starting bucket ops to source bucket
2021/12/25 08:13:36 Removing node(s): [] from the cluster
2021/12/25 08:13:41 Rebalance progress: 12.5
2021/12/25 08:13:46 Rebalance progress: 12.5
2021/12/25 08:13:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:13:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:13:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:13:51 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 08:13:52 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:13:57 Rebalance progress: 12.5
2021/12/25 08:14:02 Rebalance progress: 12.5
2021/12/25 08:14:07 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:14:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:14:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:14:07 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:14:12 Rebalance progress: 12.5
2021/12/25 08:14:17 Rebalance progress: 12.5
2021/12/25 08:14:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:14:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:14:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:14:27 Waiting for dst bucket item count to get to: 50000 curr count: 25648
2021/12/25 08:14:32 Waiting for dst bucket item count to get to: 50000 curr count: 25648
2021/12/25 08:14:37 Waiting for dst bucket item count to get to: 50000 curr count: 36147
2021/12/25 08:14:42 Waiting for dst bucket item count to get to: 50000 curr count: 36147
2021/12/25 08:14:47 Waiting for dst bucket item count to get to: 50000 curr count: 46679
2021/12/25 08:14:52 Waiting for dst bucket item count to get to: 50000 curr count: 46679
2021/12/25 08:14:52 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:14:52 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:14:57 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:02 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:07 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:12 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:17 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:22 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/25 08:15:22 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/25 08:15:22 No eventing-con process running
2021/12/25 08:15:22 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb deleting in background"
}
2021/12/25 08:15:24 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:15:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:15:34 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:15:39 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:15:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:15:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:15:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:15:51 Waiting for dst bucket item count to get to: 0 curr count: 48122
2021/12/25 08:15:56 Waiting for dst bucket item count to get to: 0 curr count: 48122
2021/12/25 08:15:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingSwapReb (189.70s)
=== RUN   TestDuplicateTimerCreationKVRebIn
2021/12/25 08:16:06 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:16:06 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/25 08:16:06 No eventing-con process running
2021/12/25 08:16:06 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebIn not found"
}
2021/12/25 08:16:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:16:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:16:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:16:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:16:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:16:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:16:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:16:24 Deploying app: TestDuplicateTimerCreationKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:16:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn stored in temp store"
}
2021/12/25 08:16:31 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/25 08:16:36 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/25 08:16:41 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/25 08:16:46 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/25 08:16:46 App: TestDuplicateTimerCreationKVRebIn got deployed
2021/12/25 08:16:46 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:16:46 Starting bucket ops to source bucket
2021/12/25 08:16:46 Removing node(s): [] from the cluster
2021/12/25 08:16:51 Rebalance progress: 12.5
2021/12/25 08:16:56 Rebalance progress: 12.5
2021/12/25 08:17:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:17:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:17:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:17:01 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:17:06 Rebalance progress: 12.5
2021/12/25 08:17:11 Rebalance progress: 12.5
2021/12/25 08:17:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:17:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:17:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:17:16 Waiting for src bucket item count to get to: 50000 curr count: 29271
2021/12/25 08:17:21 Waiting for src bucket item count to get to: 50000 curr count: 40387
2021/12/25 08:17:26 Waiting for src bucket item count to get to: 50000 curr count: 40387
2021/12/25 08:17:36 Waiting for dst bucket item count to get to: 50000 curr count: 33480
2021/12/25 08:17:41 Waiting for dst bucket item count to get to: 50000 curr count: 33480
2021/12/25 08:17:46 Waiting for dst bucket item count to get to: 50000 curr count: 40664
2021/12/25 08:17:51 Waiting for dst bucket item count to get to: 50000 curr count: 40664
2021/12/25 08:17:51 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:17:51 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:17:57 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:18:02 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:18:07 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:18:12 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:18:17 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/25 08:18:17 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/25 08:18:17 No eventing-con process running
2021/12/25 08:18:17 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn deleting in background"
}
2021/12/25 08:18:19 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:18:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:18:29 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:18:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:18:31 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:18:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:18:41 Waiting for dst bucket item count to get to: 0 curr count: 33215
2021/12/25 08:18:46 Waiting for dst bucket item count to get to: 0 curr count: 33215
2021/12/25 08:18:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebIn (169.61s)
=== RUN   TestDuplicateTimerCreationKVRebOut
2021/12/25 08:18:51 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:18:51 Removing node(s): [] from the cluster
2021/12/25 08:18:56 Rebalance progress: 100
2021/12/25 08:19:01 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:19:01 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/25 08:19:01 No eventing-con process running
2021/12/25 08:19:01 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebOut not found"
}
2021/12/25 08:19:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:19:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:19:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:19:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:19:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:19:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:19:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:19:19 Deploying app: TestDuplicateTimerCreationKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:19:21 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut stored in temp store"
}
2021/12/25 08:19:26 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/25 08:19:31 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/25 08:19:36 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/25 08:19:41 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/25 08:19:41 App: TestDuplicateTimerCreationKVRebOut got deployed
2021/12/25 08:19:41 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:19:41 Starting bucket ops to source bucket
2021/12/25 08:19:46 Rebalance progress: 12.5
2021/12/25 08:19:51 Rebalance progress: 12.5
2021/12/25 08:19:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:19:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:19:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:19:56 Waiting for src bucket item count to get to: 50000 curr count: 11307
2021/12/25 08:20:01 Waiting for src bucket item count to get to: 50000 curr count: 25457
2021/12/25 08:20:06 Waiting for src bucket item count to get to: 50000 curr count: 25457
2021/12/25 08:20:11 Waiting for src bucket item count to get to: 50000 curr count: 36574
2021/12/25 08:20:16 Waiting for src bucket item count to get to: 50000 curr count: 36574
2021/12/25 08:20:21 Waiting for src bucket item count to get to: 50000 curr count: 46252
2021/12/25 08:20:26 Waiting for src bucket item count to get to: 50000 curr count: 46252
2021/12/25 08:20:36 Waiting for dst bucket item count to get to: 50000 curr count: 36834
2021/12/25 08:20:41 Waiting for dst bucket item count to get to: 50000 curr count: 36834
2021/12/25 08:20:46 Waiting for dst bucket item count to get to: 50000 curr count: 36834
2021/12/25 08:20:51 Waiting for dst bucket item count to get to: 50000 curr count: 49383
2021/12/25 08:20:56 Waiting for dst bucket item count to get to: 50000 curr count: 49383
2021/12/25 08:20:56 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:20:56 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:21:01 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:21:06 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:21:11 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:21:16 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:21:21 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/25 08:21:21 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/25 08:21:21 No eventing-con process running
2021/12/25 08:21:22 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut deleting in background"
}
2021/12/25 08:21:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:21:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:21:33 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:21:38 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:21:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:21:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:21:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:21:50 Waiting for dst bucket item count to get to: 0 curr count: 9756
2021/12/25 08:21:55 Waiting for dst bucket item count to get to: 0 curr count: 9756
2021/12/25 08:21:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebOut (189.54s)
=== RUN   TestDuplicateTimerCreationKVSwapReb
2021/12/25 08:22:05 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:22:05 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/25 08:22:05 No eventing-con process running
2021/12/25 08:22:05 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVSwapReb not found"
}
2021/12/25 08:22:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:22:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:22:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:22:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:22:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:22:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:22:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:22:24 Deploying app: TestDuplicateTimerCreationKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:22:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb stored in temp store"
}
2021/12/25 08:22:30 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/25 08:22:35 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/25 08:22:40 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/25 08:22:45 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/25 08:22:45 App: TestDuplicateTimerCreationKVSwapReb got deployed
2021/12/25 08:22:45 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:22:45 Starting bucket ops to source bucket
2021/12/25 08:22:46 Removing node(s): [] from the cluster
2021/12/25 08:22:51 Rebalance progress: 12.5
2021/12/25 08:22:56 Rebalance progress: 12.5
2021/12/25 08:23:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:23:01 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:23:01 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:23:06 Rebalance progress: 12.5
2021/12/25 08:23:11 Rebalance progress: 12.5
2021/12/25 08:23:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:23:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:23:22 Rebalance progress: 12.5
2021/12/25 08:23:27 Rebalance progress: 12.5
2021/12/25 08:23:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:23:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:23:37 Waiting for dst bucket item count to get to: 50000 curr count: 31356
2021/12/25 08:23:42 Waiting for dst bucket item count to get to: 50000 curr count: 31356
2021/12/25 08:23:47 Waiting for dst bucket item count to get to: 50000 curr count: 38166
2021/12/25 08:23:52 Waiting for dst bucket item count to get to: 50000 curr count: 38166
2021/12/25 08:23:52 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:23:52 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:23:57 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:24:02 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:24:07 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:24:12 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:24:17 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/25 08:24:17 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/25 08:24:17 No eventing-con process running
2021/12/25 08:24:17 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb deleting in background"
}
2021/12/25 08:24:19 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:24:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:24:29 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:24:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:24:31 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:24:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:24:41 Waiting for dst bucket item count to get to: 0 curr count: 36978
2021/12/25 08:24:46 Waiting for dst bucket item count to get to: 0 curr count: 36978
2021/12/25 08:24:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVSwapReb (170.59s)
=== RUN   TestDuplicateTimerFiringEventingRebIn
2021/12/25 08:24:56 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:24:56 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/25 08:24:56 No eventing-con process running
2021/12/25 08:24:56 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebIn not found"
}
2021/12/25 08:24:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:25:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:25:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:25:04 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:25:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:25:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:25:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:25:14 Deploying app: TestDuplicateTimerFiringEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:25:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn stored in temp store"
}
2021/12/25 08:25:20 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/25 08:25:25 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/25 08:25:30 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/25 08:25:35 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/25 08:25:35 App: TestDuplicateTimerFiringEventingRebIn got deployed
2021/12/25 08:25:35 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:25:35 Starting bucket ops to source bucket
2021/12/25 08:25:36 Removing node(s): [] from the cluster
2021/12/25 08:25:41 Rebalance progress: 12.5
2021/12/25 08:25:46 Rebalance progress: 12.5
2021/12/25 08:25:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:25:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:25:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:25:51 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:25:56 Rebalance progress: 12.5
2021/12/25 08:26:01 Rebalance progress: 12.5
2021/12/25 08:26:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:26:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:26:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:26:06 Waiting for src bucket item count to get to: 50000 curr count: 29100
2021/12/25 08:26:11 Waiting for src bucket item count to get to: 50000 curr count: 40485
2021/12/25 08:26:16 Waiting for src bucket item count to get to: 50000 curr count: 40485
2021/12/25 08:26:26 Waiting for dst bucket item count to get to: 50000 curr count: 33856
2021/12/25 08:26:31 Waiting for dst bucket item count to get to: 50000 curr count: 33856
2021/12/25 08:26:36 Waiting for dst bucket item count to get to: 50000 curr count: 43426
2021/12/25 08:26:41 Waiting for dst bucket item count to get to: 50000 curr count: 43426
2021/12/25 08:26:41 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:26:41 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:26:46 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:26:51 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:26:56 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:27:02 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:27:07 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/25 08:27:07 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/25 08:27:07 No eventing-con process running
2021/12/25 08:27:07 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn deleting in background"
}
2021/12/25 08:27:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:27:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:27:18 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:27:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:27:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:27:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:27:30 Waiting for dst bucket item count to get to: 0 curr count: 17051
2021/12/25 08:27:35 Waiting for dst bucket item count to get to: 0 curr count: 17051
2021/12/25 08:27:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebIn (169.55s)
=== RUN   TestDuplicateTimerFiringEventingRebOut
2021/12/25 08:27:40 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:27:41 Removing node(s): [] from the cluster
2021/12/25 08:27:46 Rebalance progress: 100
2021/12/25 08:27:51 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:27:51 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/25 08:27:51 No eventing-con process running
2021/12/25 08:27:51 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebOut not found"
}
2021/12/25 08:27:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:27:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:27:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:27:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:28:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:28:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:28:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:28:09 Deploying app: TestDuplicateTimerFiringEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:28:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut stored in temp store"
}
2021/12/25 08:28:15 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/25 08:28:20 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/25 08:28:25 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/25 08:28:30 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/25 08:28:30 App: TestDuplicateTimerFiringEventingRebOut got deployed
2021/12/25 08:28:30 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:28:30 Starting bucket ops to source bucket
2021/12/25 08:28:36 Rebalance progress: 12.5
2021/12/25 08:28:41 Rebalance progress: 12.5
2021/12/25 08:28:46 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:28:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:28:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:28:46 Waiting for src bucket item count to get to: 50000 curr count: 11800
2021/12/25 08:28:51 Waiting for src bucket item count to get to: 50000 curr count: 25311
2021/12/25 08:28:56 Waiting for src bucket item count to get to: 50000 curr count: 25311
2021/12/25 08:29:01 Waiting for src bucket item count to get to: 50000 curr count: 35777
2021/12/25 08:29:06 Waiting for src bucket item count to get to: 50000 curr count: 35777
2021/12/25 08:29:11 Waiting for src bucket item count to get to: 50000 curr count: 47407
2021/12/25 08:29:16 Waiting for src bucket item count to get to: 50000 curr count: 47407
2021/12/25 08:29:21 Waiting for src bucket item count to get to: 50000 curr count: 47407
2021/12/25 08:29:31 Waiting for dst bucket item count to get to: 50000 curr count: 36911
2021/12/25 08:29:36 Waiting for dst bucket item count to get to: 50000 curr count: 36911
2021/12/25 08:29:36 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:29:36 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:29:41 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:29:46 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:29:51 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:29:56 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:30:01 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/25 08:30:01 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/25 08:30:01 No eventing-con process running
2021/12/25 08:30:01 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut deleting in background"
}
2021/12/25 08:30:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:30:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:30:13 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:30:18 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:30:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:30:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:30:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:30:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebOut (169.54s)
=== RUN   TestDuplicateTimerFiringEventingSwapReb
2021/12/25 08:30:35 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:30:35 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/25 08:30:35 No eventing-con process running
2021/12/25 08:30:35 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingSwapReb not found"
}
2021/12/25 08:30:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:30:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:30:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:30:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:30:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:30:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:30:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:30:53 Deploying app: TestDuplicateTimerFiringEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:30:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb stored in temp store"
}
2021/12/25 08:30:59 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/25 08:31:04 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/25 08:31:10 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/25 08:31:15 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/25 08:31:15 App: TestDuplicateTimerFiringEventingSwapReb got deployed
2021/12/25 08:31:15 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/25 08:31:15 Starting bucket ops to source bucket
2021/12/25 08:31:15 Removing node(s): [] from the cluster
2021/12/25 08:31:20 Rebalance progress: 12.5
2021/12/25 08:31:25 Rebalance progress: 12.5
2021/12/25 08:31:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:31:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:31:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:31:30 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 08:31:31 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:31:36 Rebalance progress: 12.5
2021/12/25 08:31:41 Rebalance progress: 12.5
2021/12/25 08:31:46 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:31:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:31:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:31:46 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:31:51 Rebalance progress: 12.5
2021/12/25 08:31:56 Rebalance progress: 12.5
2021/12/25 08:32:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:32:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:32:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:32:01 Waiting for src bucket item count to get to: 50000 curr count: 49741
2021/12/25 08:32:06 Waiting for src bucket item count to get to: 50000 curr count: 49741
2021/12/25 08:32:16 Waiting for dst bucket item count to get to: 50000 curr count: 46456
2021/12/25 08:32:21 Waiting for dst bucket item count to get to: 50000 curr count: 46456
2021/12/25 08:32:21 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:32:21 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:32:26 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:32:31 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:32:36 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:32:42 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:32:47 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/25 08:32:47 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/25 08:32:47 No eventing-con process running
2021/12/25 08:32:47 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb deleting in background"
}
2021/12/25 08:32:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:32:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:32:59 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:32:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:33:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:33:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:33:10 Waiting for dst bucket item count to get to: 0 curr count: 21112
2021/12/25 08:33:15 Waiting for dst bucket item count to get to: 0 curr count: 21112
2021/12/25 08:33:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingSwapReb (170.54s)
=== RUN   TestDuplicateTimerFiringKVRebIn
2021/12/25 08:33:25 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:33:25 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/25 08:33:25 No eventing-con process running
2021/12/25 08:33:25 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebIn not found"
}
2021/12/25 08:33:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:33:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:33:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:33:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:33:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:33:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:33:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:33:44 Deploying app: TestDuplicateTimerFiringKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:33:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn stored in temp store"
}
2021/12/25 08:33:50 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/25 08:33:55 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/25 08:34:00 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/25 08:34:05 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/25 08:34:05 App: TestDuplicateTimerFiringKVRebIn got deployed
2021/12/25 08:34:05 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:34:05 Starting bucket ops to source bucket
2021/12/25 08:34:05 Removing node(s): [] from the cluster
2021/12/25 08:34:11 Rebalance progress: 12.5
2021/12/25 08:34:16 Rebalance progress: 12.5
2021/12/25 08:34:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:34:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:34:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:34:21 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:34:26 Rebalance progress: 12.5
2021/12/25 08:34:31 Rebalance progress: 12.5
2021/12/25 08:34:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:34:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:34:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:34:36 Waiting for src bucket item count to get to: 50000 curr count: 28798
2021/12/25 08:34:41 Waiting for src bucket item count to get to: 50000 curr count: 40920
2021/12/25 08:34:46 Waiting for src bucket item count to get to: 50000 curr count: 40920
2021/12/25 08:34:56 Waiting for dst bucket item count to get to: 50000 curr count: 36819
2021/12/25 08:35:01 Waiting for dst bucket item count to get to: 50000 curr count: 36819
2021/12/25 08:35:06 Waiting for dst bucket item count to get to: 50000 curr count: 44987
2021/12/25 08:35:11 Waiting for dst bucket item count to get to: 50000 curr count: 44987
2021/12/25 08:35:11 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:35:11 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:35:16 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:35:21 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:35:26 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:35:31 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:35:36 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/25 08:35:36 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/25 08:35:36 No eventing-con process running
2021/12/25 08:35:36 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn deleting in background"
}
2021/12/25 08:35:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:35:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:35:48 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:35:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:35:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:35:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:36:00 Waiting for dst bucket item count to get to: 0 curr count: 2676
2021/12/25 08:36:05 Waiting for dst bucket item count to get to: 0 curr count: 2676
2021/12/25 08:36:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebIn (169.60s)
=== RUN   TestDuplicateTimerFiringKVRebOut
2021/12/25 08:36:10 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:36:10 Removing node(s): [] from the cluster
2021/12/25 08:36:15 Rebalance progress: 100
2021/12/25 08:36:20 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:36:20 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/25 08:36:20 No eventing-con process running
2021/12/25 08:36:20 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebOut not found"
}
2021/12/25 08:36:22 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:36:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:36:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:36:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:36:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:36:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:36:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:36:39 Deploying app: TestDuplicateTimerFiringKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:36:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut stored in temp store"
}
2021/12/25 08:36:45 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/25 08:36:50 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/25 08:36:55 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/25 08:37:00 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/25 08:37:00 App: TestDuplicateTimerFiringKVRebOut got deployed
2021/12/25 08:37:00 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:37:00 Starting bucket ops to source bucket
2021/12/25 08:37:05 Rebalance progress: 12.5
2021/12/25 08:37:10 Rebalance progress: 12.5
2021/12/25 08:37:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:37:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:37:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:37:15 Waiting for src bucket item count to get to: 50000 curr count: 12757
2021/12/25 08:37:20 Waiting for src bucket item count to get to: 50000 curr count: 25879
2021/12/25 08:37:25 Waiting for src bucket item count to get to: 50000 curr count: 25879
2021/12/25 08:37:30 Waiting for src bucket item count to get to: 50000 curr count: 35131
2021/12/25 08:37:35 Waiting for src bucket item count to get to: 50000 curr count: 35131
2021/12/25 08:37:40 Waiting for src bucket item count to get to: 50000 curr count: 46375
2021/12/25 08:37:45 Waiting for src bucket item count to get to: 50000 curr count: 46375
2021/12/25 08:37:50 Waiting for src bucket item count to get to: 50000 curr count: 46375
2021/12/25 08:38:00 Waiting for dst bucket item count to get to: 50000 curr count: 41277
2021/12/25 08:38:05 Waiting for dst bucket item count to get to: 50000 curr count: 41277
2021/12/25 08:38:05 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:38:05 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:38:10 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:15 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:20 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:25 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:31 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:36 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/25 08:38:36 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/25 08:38:36 No eventing-con process running
2021/12/25 08:38:36 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut deleting in background"
}
2021/12/25 08:38:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:38:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:38:47 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:38:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:38:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:38:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:38:59 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:38:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebOut (174.31s)
=== RUN   TestDuplicateTimerFiringKVSwapReb
2021/12/25 08:39:09 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:39:09 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/25 08:39:09 No eventing-con process running
2021/12/25 08:39:09 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVSwapReb not found"
}
2021/12/25 08:39:11 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:39:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:39:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:39:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:39:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:39:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:39:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:39:28 Deploying app: TestDuplicateTimerFiringKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:39:29 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb stored in temp store"
}
2021/12/25 08:39:34 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/25 08:39:39 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/25 08:39:44 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/25 08:39:49 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/25 08:39:49 App: TestDuplicateTimerFiringKVSwapReb got deployed
2021/12/25 08:39:49 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:39:49 Starting bucket ops to source bucket
2021/12/25 08:39:50 Removing node(s): [] from the cluster
2021/12/25 08:39:55 Rebalance progress: 12.5
2021/12/25 08:40:00 Rebalance progress: 12.5
2021/12/25 08:40:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:40:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:40:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:40:10 Rebalance progress: 12.5
2021/12/25 08:40:15 Rebalance progress: 12.5
2021/12/25 08:40:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:40:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:40:25 Rebalance progress: 12.5
2021/12/25 08:40:30 Rebalance progress: 12.5
2021/12/25 08:40:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:40:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:40:35 Waiting for src bucket item count to get to: 50000 curr count: 46948
2021/12/25 08:40:40 Waiting for src bucket item count to get to: 50000 curr count: 46948
2021/12/25 08:40:50 Waiting for dst bucket item count to get to: 50000 curr count: 42174
2021/12/25 08:40:55 Waiting for dst bucket item count to get to: 50000 curr count: 42174
2021/12/25 08:40:55 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/25 08:40:55 Post SET ops. Expected item count: 50000 got 50000
2021/12/25 08:41:00 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:41:06 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:41:11 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:41:16 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:41:21 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/25 08:41:21 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/25 08:41:21 No eventing-con process running
2021/12/25 08:41:21 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb deleting in background"
}
2021/12/25 08:41:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:41:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:41:33 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:41:38 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/25 08:41:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:41:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:41:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:41:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVSwapReb (165.32s)
=== RUN   TestKVRebOnUpdateBucketOpOneByOne
2021/12/25 08:41:55 Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:41:55 App: bucket_op_on_update got un-deployed
2021/12/25 08:41:55 No eventing-con process running
2021/12/25 08:41:55 Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:41:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:42:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:42:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:42:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:42:13 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:42:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 08:42:19 Starting bucket ops to source bucket
2021/12/25 08:42:24 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:42:29 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:42:34 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:42:34 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:42:34 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:42:35 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:42:40 Rebalance progress: 12.5
2021/12/25 08:42:45 Rebalance progress: 12.5
2021/12/25 08:42:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:42:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:42:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:42:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:42:55 Rebalance progress: 12.5
2021/12/25 08:43:00 Rebalance progress: 12.5
2021/12/25 08:43:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:43:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/25 08:43:05 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:43:10 Rebalance progress: 12.5
2021/12/25 08:43:15 Rebalance progress: 12.5
2021/12/25 08:43:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:43:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:43:25 Rebalance progress: 12.5
2021/12/25 08:43:30 Rebalance progress: 12.5
2021/12/25 08:43:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:43:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:43:41 Rebalance progress: 12.5
2021/12/25 08:43:46 Rebalance progress: 12.5
2021/12/25 08:43:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:43:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:43:51 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 08:43:56 Rebalance progress: 12.5
2021/12/25 08:44:01 Rebalance progress: 12.5
2021/12/25 08:44:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:44:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:44:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:44:11 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:44:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:44:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:44:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:44:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:44:31 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:44:31 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:44:31 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:44:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:44:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:44:43 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10622
2021/12/25 08:44:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10622
2021/12/25 08:44:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:44:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:44:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:44:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpOneByOne (190.17s)
=== RUN   TestKVForceFailover
2021/12/25 08:45:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:45:10 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:45:10 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:45:10 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:45:12 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:45:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:45:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:45:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:45:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:45:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:45:23 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:45:24 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:45:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:45:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:45:29 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:45:30 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 08:45:30 Starting bucket ops to source bucket
2021/12/25 08:45:35 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:45:40 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:45:45 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:45:50 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:45:50 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:45:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:45:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:45:50 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/25 08:46:10 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/25 08:46:10 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/25 08:46:15 Rebalance progress: 12.5
2021/12/25 08:46:20 Rebalance progress: 12.5
2021/12/25 08:46:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:46:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:46:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:46:25 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/25 08:46:30 Rebalance progress: 12.5
2021/12/25 08:46:35 Rebalance progress: 12.5
2021/12/25 08:46:40 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:46:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:46:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:46:45 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:46:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:46:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:47:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:47:06 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:47:06 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:47:06 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:47:06 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:47:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:47:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:47:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7037
2021/12/25 08:47:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:47:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:47:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:47:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVForceFailover (149.76s)
=== RUN   TestKVHardFailoverRecovery
2021/12/25 08:47:34 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:47:35 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:47:35 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:47:35 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:47:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:47:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:47:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:47:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:47:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:47:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:47:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:47:53 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:47:54 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 08:47:59 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 08:48:04 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:48:09 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:48:14 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:48:14 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:48:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:48:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:48:14 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:48:15 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:48:20 Rebalance progress: 12.5
2021/12/25 08:48:25 Rebalance progress: 12.5
2021/12/25 08:48:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:48:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:48:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:48:30 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/25 08:49:30 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/25 08:49:35 Rebalance progress: 12.5
2021/12/25 08:49:40 Rebalance progress: 12.5
2021/12/25 08:49:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:49:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:49:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:49:45 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:49:50 Rebalance progress: 12.5
2021/12/25 08:49:55 Rebalance progress: 12.5
2021/12/25 08:50:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:50:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:50:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:50:05 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:26 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:50:26 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:50:26 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:50:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:50:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:50:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12068
2021/12/25 08:50:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:50:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:50:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:50:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12068
2021/12/25 08:50:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVHardFailoverRecovery (204.80s)
=== RUN   TestKVRebOnUpdateBucketOpAllAtOnce
2021/12/25 08:50:59 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:50:59 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:50:59 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:50:59 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:51:01 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:06 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:51:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:51:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:51:13 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:51:18 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:51:19 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 08:51:24 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 08:51:29 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:51:34 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:51:39 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:51:39 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:51:39 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:51:39 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:51:40 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/25 08:51:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:51:45 Rebalance progress: 12.5
2021/12/25 08:51:50 Rebalance progress: 12.5
2021/12/25 08:51:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:51:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:51:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/25 08:52:00 Rebalance progress: 12.5
2021/12/25 08:52:05 Rebalance progress: 12.5
2021/12/25 08:52:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:52:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:52:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:52:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:52:20 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:52:25 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:52:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:52:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:52:36 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:52:36 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:52:36 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:52:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:52:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:52:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4602
2021/12/25 08:52:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:52:49 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:52:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:52:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4602
2021/12/25 08:52:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpAllAtOnce (129.93s)
=== RUN   TestKVSwapRebOnUpdateBucketOp
2021/12/25 08:53:09 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:53:09 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:53:09 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:53:09 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:53:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:53:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:53:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:53:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:53:23 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:53:24 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 08:53:29 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 08:53:34 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:53:39 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:53:44 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:53:44 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:53:44 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:53:44 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:53:49 Rebalance progress: 12.5
2021/12/25 08:53:54 Rebalance progress: 12.5
2021/12/25 08:53:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:53:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:53:59 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:54:00 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:54:05 Rebalance progress: 12.5
2021/12/25 08:54:10 Rebalance progress: 12.5
2021/12/25 08:54:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:54:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:54:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:54:15 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:54:20 Rebalance progress: 12.5
2021/12/25 08:54:25 Rebalance progress: 12.5
2021/12/25 08:54:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:54:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:54:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:54:35 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:54:40 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:54:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:54:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:54:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:54:56 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:54:56 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:54:56 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:54:58 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:03 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6092
2021/12/25 08:55:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:55:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:55:19 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:55:19 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:55:19 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:55:19 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:55:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:55:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebOnUpdateBucketOp (153.63s)
=== RUN   TestKVRebalanceWithMultipleHandlers
2021/12/25 08:55:43 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:55:43 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:55:43 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:55:43 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 08:55:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:55:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:55:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:55:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:56:01 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:56:01 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/25 08:56:01 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:56:01 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/25 08:56:03 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:56:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:56:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:56:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:56:15 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:56:16 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:56:16 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 08:56:17 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/25 08:56:22 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 08:56:27 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:56:32 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:56:37 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 08:56:37 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/25 08:56:42 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 08:56:47 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 08:56:52 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 08:56:52 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:56:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:56:52 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 08:56:53 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:56:58 Rebalance progress: 12.5
2021/12/25 08:57:03 Rebalance progress: 12.5
2021/12/25 08:57:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:57:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 08:57:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:57:13 Rebalance progress: 12.5
2021/12/25 08:57:18 Rebalance progress: 12.5
2021/12/25 08:57:23 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:57:23 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/25 08:57:23 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 08:57:28 Rebalance progress: 12.5
2021/12/25 08:57:33 Rebalance progress: 12.5
2021/12/25 08:57:38 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:57:39 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 08:57:44 Rebalance progress: 12.5
2021/12/25 08:57:49 Rebalance progress: 12.5
2021/12/25 08:57:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:57:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:57:54 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 08:57:59 Rebalance progress: 12.5
2021/12/25 08:58:04 Rebalance progress: 12.5
2021/12/25 08:58:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:58:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:58:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:58:09 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/25 08:58:14 Rebalance progress: 12.5
2021/12/25 08:58:19 Rebalance progress: 12.5
2021/12/25 08:58:24 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:58:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:58:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 08:58:29 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:58:34 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:58:39 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:58:44 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:58:49 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 08:58:49 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 08:58:50 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:58:50 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 08:58:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:58:58 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:59:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12149
2021/12/25 08:59:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12149
2021/12/25 08:59:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:59:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:59:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:59:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4587
2021/12/25 08:59:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4587
2021/12/25 08:59:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:59:30 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:59:36 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:59:41 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:59:46 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:59:51 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 08:59:51 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/25 08:59:51 eventingSetup.go:46: No eventing-con process running
2021/12/25 08:59:51 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/25 08:59:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 08:59:58 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 08:59:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 08:59:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebalanceWithMultipleHandlers (271.64s)
=== RUN   TestKVSwapRebalanceWithMultipleHandlers
2021/12/25 09:00:14 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:00:15 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:00:15 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:00:15 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 09:00:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:00:23 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:00:33 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:00:33 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/25 09:00:33 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:00:33 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/25 09:00:35 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:40 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:00:42 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:47 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 09:00:48 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:00:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:00:48 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/25 09:00:49 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/25 09:00:54 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 09:00:59 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:01:04 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:01:09 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:01:09 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/25 09:01:14 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 09:01:19 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 09:01:24 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/25 09:01:24 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:01:24 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 09:01:24 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:01:29 Rebalance progress: 12.5
2021/12/25 09:01:34 Rebalance progress: 12.5
2021/12/25 09:01:39 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:01:39 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 09:01:40 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 09:01:45 Rebalance progress: 12.5
2021/12/25 09:01:50 Rebalance progress: 12.5
2021/12/25 09:01:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:01:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:01:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 09:02:00 Rebalance progress: 12.5
2021/12/25 09:02:05 Rebalance progress: 12.5
2021/12/25 09:02:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:02:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:02:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:02:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:02:20 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:02:25 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:02:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:02:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:02:36 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:02:36 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:02:36 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 09:02:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:02:45 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:02:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:02:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:02:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:02:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7593
2021/12/25 09:02:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:03:02 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:03:07 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:03:12 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:03:17 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:03:23 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/25 09:03:23 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/25 09:03:23 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:03:23 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/25 09:03:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:03:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:03:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:03:31 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:03:36 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:03:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebalanceWithMultipleHandlers (211.75s)
=== RUN   TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/25 09:03:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:03:46 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:03:46 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:03:46 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 09:03:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:03:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:03:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:03:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:04:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:04:05 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 09:04:06 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 09:04:11 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 09:04:16 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:04:21 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:04:26 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:04:26 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:04:26 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 09:04:26 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:04:46 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:04:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:04:57 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:05:02 Rebalance progress: 12.5
2021/12/25 09:05:07 Rebalance progress: 12.5
2021/12/25 09:05:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:05:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 09:05:12 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 09:05:17 Rebalance progress: 12.5
2021/12/25 09:05:22 Rebalance progress: 12.5
2021/12/25 09:05:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:05:27 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 09:05:33 Rebalance progress: 12.5
2021/12/25 09:05:38 Rebalance progress: 12.5
2021/12/25 09:05:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:05:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:05:48 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:05:53 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:05:58 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:06:03 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:06:08 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:06:08 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:06:08 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:06:08 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 09:06:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:06:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:06:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 502
2021/12/25 09:06:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 502
2021/12/25 09:06:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:06:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:06:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:06:37 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 9148
2021/12/25 09:06:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne (180.44s)
=== RUN   TestKVFailoverOnUpdateBucketOp
2021/12/25 09:06:47 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:06:47 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:06:47 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:06:47 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/25 09:06:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:06:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:06:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:06:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:07:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:07:05 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 09:07:06 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 09:07:11 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/25 09:07:16 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:07:21 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:07:26 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:07:26 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:07:26 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 09:07:27 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:07:32 Rebalance progress: 12.5
2021/12/25 09:07:37 Rebalance progress: 12.5
2021/12/25 09:07:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:07:42 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:07:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:07:52 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:07:57 Rebalance progress: 12.5
2021/12/25 09:08:02 Rebalance progress: 12.5
2021/12/25 09:08:07 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:08:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/25 09:08:07 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/25 09:08:12 Rebalance progress: 12.5
2021/12/25 09:08:17 Rebalance progress: 12.5
2021/12/25 09:08:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:08:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/25 09:08:27 Rebalance progress: 12.5
2021/12/25 09:08:32 Rebalance progress: 12.5
2021/12/25 09:08:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:08:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:08:43 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:08:48 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:08:53 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:08:58 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:09:03 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:09:03 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:09:03 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:09:03 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 09:09:05 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:09:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:09:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8604
2021/12/25 09:09:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:09:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:09:22 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:09:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8604
2021/12/25 09:09:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVFailoverOnUpdateBucketOp (170.28s)
=== RUN   TestBootstrapAfterKVHardFailover
2021/12/25 09:09:32 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/25 09:09:32 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/25 09:09:37 Rebalance progress: 100
2021/12/25 09:09:37 Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:10:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:10:07 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/25 09:10:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/25 09:10:13 Removing node(s): [] from the cluster
2021/12/25 09:10:13 Starting bucket ops to source bucket
2021/12/25 09:10:19 Rebalance failed. See logs for detailed reason. You can try again.
2021/12/25 09:10:24 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:10:29 Waiting for app: bucket_op_on_update to get deployed
2021/12/25 09:10:29 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:10:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:10:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/25 09:10:34 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:10:39 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:10:44 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:10:49 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:10:54 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/25 09:10:54 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/25 09:10:54 eventingSetup.go:46: No eventing-con process running
2021/12/25 09:10:54 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/25 09:10:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:11:01 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:11:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1503
2021/12/25 09:11:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/25 09:11:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/25 09:11:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/25 09:11:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1503
2021/12/25 09:11:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBootstrapAfterKVHardFailover (110.81s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	11936.531s

Rebalance tests took 3 hours, 18 minutes and 59 seconds.

Testing Succeeded


Version: versions-25.12.2021-01.32.cfg
Build Log: make-25.12.2021-01.32.log
Server Log: logs-25.12.2021-01.32.tar.gz

Finished