Building

Started building at 2021/12/23 10:21:13
Calculating base
Updating mirror
Basing run on 7.1.0-1972 457d8c0af
Updating tree for run 23.12.2021-10.21
gocb is at 635fa6c, changes since last good build: none
libcouchbase is at f6822fc, changes since last good build: none
indexing is at 8d3522b, changes since last good build: 
 8d3522b0 Merge remote-tracking branch 'couchbase/unstable' into HEAD
 766ec09f MB-46139: getMinMemory calculation is wrong when resident ratio is low
 0581a844 MB-50008 fix num_docs_queued stat
 9c2258ea MB-49293 : Print logs at Error level on getIndexStatus failure
kv_engine is at cda4d73, changes since last good build: 
 cda4d736b MB-35297: Use getVbucketId consistently in backfill classes
 68a37ed9b MB-35297: Use Expects when obtaining the KVStore pointer
ns_server is at 8590527, changes since last good build: 
 859052753 MB-49895 Tag raw_url for redaction
couchstore is at 0bf3a4b, changes since last good build: none
couchdb is at fca2e6b, changes since last good build: none
testrunner is at 9c4b191, changes since last good build: none
Switching eventing to unstable
eventing is at d9c8767, changes since last good build: none
Switching eventing-ee to unstable
eventing-ee is at cd349c5, changes since last good build: 
 cd349c5 MB-49127 : js-evaluator RBAC
Pulling in uncommitted change 160397 at refs/changes/97/160397/3
Total 23 (delta 15), reused 17 (delta 15)
[unstable 46da7977] MB-100 : Do not fail n1ql timeout test if we are catching timeout exceptions successfully
 Author: abhijpes 
 Date: Mon Aug 30 08:24:38 2021 +0530
 1 file changed, 2 insertions(+), 2 deletions(-)
Pulling in uncommitted change 163449 at refs/changes/49/163449/1
Total 48 (delta 27), reused 39 (delta 27)
[unstable 673d423d] MB-100: collect data when test fails
 Author: AnkitPrabhu 
 Date: Tue Oct 12 14:02:31 2021 +0530
 1 file changed, 2 insertions(+)
Pulling in uncommitted change 165806 at refs/changes/06/165806/45
Total 34 (delta 25), reused 34 (delta 25)
[unstable 27851879] MB-48538: Audit log refactor
 Author: AnkitPrabhu 
 Date: Mon Dec 13 10:30:25 2021 +0530
 18 files changed, 3165 insertions(+), 2213 deletions(-)
 create mode 100644 service_manager/response/defs.go
 create mode 100644 service_manager/response/response.go
Pulling in uncommitted change 165976 at refs/changes/76/165976/19
[unstable 11f94943] MB-48538: Add new audit ids for different events
 Author: AnkitPrabhu 
 Date: Tue Nov 23 14:29:10 2021 +0530
 1 file changed, 337 insertions(+), 61 deletions(-)
Pulling in uncommitted change 166698 at refs/changes/98/166698/3
Total 4 (delta 3), reused 4 (delta 3)
[unstable 1f92d082] MB-48334 : Fail rebalance unless all functions deployed on current encryption level
 Author: abhijpes 
 Date: Wed Dec 1 14:51:29 2021 +0530
 1 file changed, 9 insertions(+)
Pulling in uncommitted change 167982 at refs/changes/82/167982/3
Total 42 (delta 32), reused 41 (delta 32)
[unstable 8efcb446] MB-50179: Use vbucket forward map for master node calculation during vbucket movement
 Author: AnkitPrabhu 
 Date: Tue Dec 21 13:32:56 2021 +0530
 3 files changed, 31 insertions(+), 27 deletions(-)
Building cmakefiles and deps
Building main product

Testing

Started testing at 2021/12/23 10:36:37
Testing mode: rebalance,n1ql,functional
Starting server
Starting server: attempt 1

Functional tests

2021/12/23 10:38:06 Started functional
Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:55444->127.0.0.1:9000: read: connection reset by peer
Quota setup Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:55444->127.0.0.1:9000: read: connection reset by peer
2021/12/23 10:38:16 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/23 10:38:22 Error in adding nodes : , response: {"otpNode":"n_1@127.0.0.1"}
2021/12/23 10:38:22 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 10:38:28 Removing node(s): [] from the cluster
2021/12/23 10:38:33 Rebalance progress: 0
2021/12/23 10:38:38 Rebalance progress: 0
2021/12/23 10:38:43 Rebalance progress: 4.358362268518518
2021/12/23 10:38:48 Rebalance progress: 8.969907407407407
2021/12/23 10:38:53 Rebalance progress: 12.2974537037037
2021/12/23 10:38:58 Rebalance progress: 17.05367476851852
2021/12/23 10:39:03 Rebalance progress: 20.41739004629629
2021/12/23 10:39:08 Rebalance progress: 24.9927662037037
2021/12/23 10:39:13 Rebalance progress: 100
2021/12/23 10:39:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:39:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:39:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:40:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:40:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:40:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:40:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:41:00 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:41:15 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:41:30 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/23 10:41:45 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
=== RUN   TestAdvancedGetOps
2021/12/23 10:42:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:42:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:42:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:42:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:43:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:43:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:43:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:43:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:44:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:44:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:44:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:44:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:45:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:45:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:45:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:45:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:hello-world Completion:0}]
2021/12/23 10:46:03 Deploying app: TestAdvancedGetOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:46:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps stored in temp store"
}
2021/12/23 10:46:09 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/23 10:46:14 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/23 10:46:19 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/23 10:46:24 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/23 10:46:24 App: TestAdvancedGetOps got deployed
2021/12/23 10:46:24 Starting bucket ops to source bucket
2021/12/23 10:46:30 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:46:35 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:46:35 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 10:46:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:46:35 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:46:40 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/23 10:46:45 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/23 10:46:50 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/23 10:46:55 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/23 10:46:55 App: TestAdvancedGetOps got un-deployed
2021/12/23 10:46:55 No eventing-con process running
2021/12/23 10:46:55 Function: TestAdvancedGetOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps deleting in background"
}
2021/12/23 10:46:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:47:06 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 10:47:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:47:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:47:18 Waiting for dst bucket item count to get to: 0 curr count: 77
2021/12/23 10:47:23 Waiting for dst bucket item count to get to: 0 curr count: 27
2021/12/23 10:47:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedGetOps (320.19s)
=== RUN   TestAdvancedInsertOps
2021/12/23 10:47:23 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:47:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/23 10:47:29 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:47:34 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:47:39 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:47:44 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:47:44 App: TestAdvancedInsertOps got deployed
2021/12/23 10:47:44 Starting bucket ops to source bucket
2021/12/23 10:47:50 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/23 10:47:55 Waiting for dst bucket item count to get to: 1024 curr count: 512
2021/12/23 10:47:55 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/23 10:47:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:47:55 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:48:00 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:48:05 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:48:10 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:48:15 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:48:16 App: TestAdvancedInsertOps got un-deployed
2021/12/23 10:48:16 No eventing-con process running
2021/12/23 10:48:16 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/23 10:48:17 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:48:27 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:48:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:48:29 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:48:39 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:48:44 Waiting for dst bucket item count to get to: 0 curr count: 512
2021/12/23 10:48:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:48:44 Testing insert operation on source bucket
2021/12/23 10:48:44 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:48:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/23 10:48:50 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:48:55 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:49:00 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:49:05 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/23 10:49:05 App: TestAdvancedInsertOps got deployed
2021/12/23 10:49:05 Starting bucket ops to source bucket
2021/12/23 10:49:11 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/23 10:49:11 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/23 10:49:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:49:11 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:49:16 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:49:21 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:49:26 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:49:31 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/23 10:49:31 App: TestAdvancedInsertOps got un-deployed
2021/12/23 10:49:31 No eventing-con process running
2021/12/23 10:49:31 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/23 10:49:33 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:49:43 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/23 10:49:48 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:49:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:49:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:49:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertOps (151.62s)
=== RUN   TestAdvancedUpsertOps
2021/12/23 10:49:55 Starting bucket ops to source bucket
2021/12/23 10:49:56 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:49:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/23 10:50:02 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:50:07 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:50:12 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:50:17 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:50:17 App: TestAdvancedUpsertOps got deployed
2021/12/23 10:50:17 Starting bucket ops to source bucket
2021/12/23 10:50:23 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:50:28 Waiting for dst bucket item count to get to: 0 curr count: 272
2021/12/23 10:50:33 Waiting for dst bucket item count to get to: 0 curr count: 272
2021/12/23 10:50:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:50:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:50:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:50:38 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:50:43 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:50:48 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:50:53 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:50:53 App: TestAdvancedUpsertOps got un-deployed
2021/12/23 10:50:53 No eventing-con process running
2021/12/23 10:50:53 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/23 10:50:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:51:05 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:51:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:51:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:51:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:51:11 Testing upsert operation on source bucket
2021/12/23 10:51:11 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:51:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/23 10:51:18 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:51:23 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:51:28 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:51:33 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/23 10:51:33 App: TestAdvancedUpsertOps got deployed
2021/12/23 10:51:33 Starting bucket ops to source bucket
2021/12/23 10:51:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:51:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:51:34 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:51:39 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:51:44 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:51:49 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:51:54 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/23 10:51:54 App: TestAdvancedUpsertOps got un-deployed
2021/12/23 10:51:54 No eventing-con process running
2021/12/23 10:51:54 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/23 10:51:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:52:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:52:03 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:52:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedUpsertOps (132.92s)
=== RUN   TestAdvancedReplaceOps
2021/12/23 10:52:08 Starting bucket ops to source bucket
2021/12/23 10:52:09 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:52:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/23 10:52:15 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:52:20 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:52:25 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:52:30 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:52:30 App: TestAdvancedReplaceOps got deployed
2021/12/23 10:52:30 Starting bucket ops to source bucket
2021/12/23 10:52:36 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:52:41 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 10:52:46 Waiting for dst bucket item count to get to: 0 curr count: 512
2021/12/23 10:52:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:52:46 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:52:46 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:52:51 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:52:56 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:53:01 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:53:06 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:53:06 App: TestAdvancedReplaceOps got un-deployed
2021/12/23 10:53:06 No eventing-con process running
2021/12/23 10:53:06 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/23 10:53:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:53:18 Waiting for dst bucket item count to get to: 0 curr count: 677
2021/12/23 10:53:23 Waiting for dst bucket item count to get to: 0 curr count: 165
2021/12/23 10:53:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:53:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:53:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:53:30 Testing upsert operation on source bucket
2021/12/23 10:53:30 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:53:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/23 10:53:36 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:53:41 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:53:46 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:53:51 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/23 10:53:51 App: TestAdvancedReplaceOps got deployed
2021/12/23 10:53:51 Starting bucket ops to source bucket
2021/12/23 10:53:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:53:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:53:52 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:53:57 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:54:02 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:54:07 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:54:12 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/23 10:54:12 App: TestAdvancedReplaceOps got un-deployed
2021/12/23 10:54:12 No eventing-con process running
2021/12/23 10:54:12 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/23 10:54:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:54:24 Waiting for dst bucket item count to get to: 0 curr count: 354
2021/12/23 10:54:29 Waiting for dst bucket item count to get to: 0 curr count: 164
2021/12/23 10:54:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:54:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:54:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedReplaceOps (147.78s)
=== RUN   TestAdvancedDeleteOps
2021/12/23 10:54:36 Starting bucket ops to source bucket
2021/12/23 10:54:36 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:54:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/23 10:54:42 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:54:47 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:54:52 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:54:57 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:54:57 App: TestAdvancedDeleteOps got deployed
2021/12/23 10:54:57 Starting bucket ops to source bucket
2021/12/23 10:55:02 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 10:55:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:55:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:55:02 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:55:07 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:55:12 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:55:17 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:55:22 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:55:22 App: TestAdvancedDeleteOps got un-deployed
2021/12/23 10:55:22 No eventing-con process running
2021/12/23 10:55:22 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/23 10:55:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:55:34 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 10:55:39 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 10:55:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:55:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:55:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:55:46 Testing delete operation on source bucket
2021/12/23 10:55:46 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:55:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/23 10:55:52 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:55:57 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:56:02 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:56:07 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/23 10:56:07 App: TestAdvancedDeleteOps got deployed
2021/12/23 10:56:07 Starting bucket ops to source bucket
2021/12/23 10:56:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:56:07 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:56:07 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:56:12 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:56:17 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:56:22 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:56:27 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/23 10:56:27 App: TestAdvancedDeleteOps got un-deployed
2021/12/23 10:56:27 No eventing-con process running
2021/12/23 10:56:27 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/23 10:56:29 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:56:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:56:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:56:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedDeleteOps (125.85s)
=== RUN   TestEnoentAdvancedGet
2021/12/23 10:56:41 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:56:43 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/23 10:56:48 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:56:53 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:56:58 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:57:03 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:57:03 App: TestEnoentAdvancedGet got deployed
2021/12/23 10:57:03 Starting bucket ops to source bucket
2021/12/23 10:57:08 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:57:13 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:57:13 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 10:57:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:57:13 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:57:18 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:57:23 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:57:28 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:57:33 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:57:33 App: TestEnoentAdvancedGet got un-deployed
2021/12/23 10:57:33 No eventing-con process running
2021/12/23 10:57:33 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/23 10:57:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:57:45 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 10:57:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:57:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:57:56 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 10:57:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:57:56 Testing get enoent operation on source bucket
2021/12/23 10:57:56 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:57:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/23 10:58:03 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:58:08 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:58:13 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:58:18 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/23 10:58:18 App: TestEnoentAdvancedGet got deployed
2021/12/23 10:58:18 Starting bucket ops to source bucket
2021/12/23 10:58:23 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/23 10:58:28 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 10:58:33 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 10:58:33 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/23 10:58:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:58:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:58:38 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:58:43 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:58:48 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:58:53 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/23 10:58:53 App: TestEnoentAdvancedGet got un-deployed
2021/12/23 10:58:53 No eventing-con process running
2021/12/23 10:58:53 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/23 10:58:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:59:05 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/23 10:59:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 10:59:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 10:59:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedGet (150.04s)
=== RUN   TestEnoentAdvancedReplace
2021/12/23 10:59:11 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 10:59:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/23 10:59:18 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 10:59:23 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 10:59:28 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 10:59:33 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 10:59:33 App: TestEnoentAdvancedReplace got deployed
2021/12/23 10:59:33 Starting bucket ops to source bucket
2021/12/23 10:59:38 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:59:43 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 10:59:43 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 10:59:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 10:59:43 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 10:59:48 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 10:59:53 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 10:59:58 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 11:00:03 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 11:00:03 App: TestEnoentAdvancedReplace got un-deployed
2021/12/23 11:00:03 No eventing-con process running
2021/12/23 11:00:03 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/23 11:00:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:00:15 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:00:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:00:16 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:00:27 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:00:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:00:27 Testing get enoent operation on source bucket
2021/12/23 11:00:27 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:00:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/23 11:00:33 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 11:00:38 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 11:00:43 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 11:00:48 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/23 11:00:48 App: TestEnoentAdvancedReplace got deployed
2021/12/23 11:00:48 Starting bucket ops to source bucket
2021/12/23 11:00:53 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/23 11:00:58 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 11:01:03 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 11:01:03 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/23 11:01:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:01:03 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:01:08 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 11:01:13 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 11:01:18 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/23 11:01:18 App: TestEnoentAdvancedReplace got un-deployed
2021/12/23 11:01:18 No eventing-con process running
2021/12/23 11:01:18 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/23 11:01:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:01:30 Waiting for dst bucket item count to get to: 0 curr count: 115
2021/12/23 11:01:35 Waiting for dst bucket item count to get to: 0 curr count: 115
2021/12/23 11:01:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:01:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:01:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedReplace (150.26s)
=== RUN   TestEnoentAdvancedDelete
2021/12/23 11:01:42 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:01:43 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/23 11:01:48 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:01:53 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:01:58 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:02:03 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:02:03 App: TestEnoentAdvancedDelete got deployed
2021/12/23 11:02:03 Starting bucket ops to source bucket
2021/12/23 11:02:08 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:02:13 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:02:13 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:02:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:02:13 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:02:18 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:02:23 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:02:28 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:02:33 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:02:33 App: TestEnoentAdvancedDelete got un-deployed
2021/12/23 11:02:33 No eventing-con process running
2021/12/23 11:02:33 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/23 11:02:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:02:45 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:02:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:02:47 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:02:57 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:02:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:02:57 Testing get enoent operation on source bucket
2021/12/23 11:02:57 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:02:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/23 11:03:03 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:03:08 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:03:13 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:03:18 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/23 11:03:18 App: TestEnoentAdvancedDelete got deployed
2021/12/23 11:03:18 Starting bucket ops to source bucket
2021/12/23 11:03:23 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/23 11:03:28 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 11:03:33 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/23 11:03:33 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/23 11:03:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:03:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:03:38 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:03:43 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:03:48 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:03:53 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/23 11:03:53 App: TestEnoentAdvancedDelete got un-deployed
2021/12/23 11:03:53 No eventing-con process running
2021/12/23 11:03:53 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/23 11:03:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:04:05 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/23 11:04:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:04:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:04:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedDelete (149.92s)
=== RUN   TestAdvancedInsertKeyExist
2021/12/23 11:04:12 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:04:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/23 11:04:18 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:04:23 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:04:28 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:04:33 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:04:33 App: TestAdvancedInsertKeyExist got deployed
2021/12/23 11:04:33 Starting bucket ops to source bucket
2021/12/23 11:04:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:04:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:04:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:04:38 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:04:43 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:04:48 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:04:53 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:04:53 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/23 11:04:53 No eventing-con process running
2021/12/23 11:04:53 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/23 11:04:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:05:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:05:02 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:05:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:05:07 Pumping document in the destination bucket
2021/12/23 11:05:07 Starting bucket ops to source bucket
2021/12/23 11:05:07 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:05:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/23 11:05:14 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:05:19 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:05:24 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:05:29 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/23 11:05:29 App: TestAdvancedInsertKeyExist got deployed
2021/12/23 11:05:29 Starting bucket ops to source bucket
2021/12/23 11:05:34 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:05:39 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:05:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:05:39 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:05:39 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:05:44 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:05:49 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:05:54 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:05:59 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/23 11:05:59 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/23 11:05:59 No eventing-con process running
2021/12/23 11:05:59 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/23 11:06:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:06:11 Waiting for dst bucket item count to get to: 0 curr count: 86
2021/12/23 11:06:16 Waiting for dst bucket item count to get to: 0 curr count: 86
2021/12/23 11:06:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:06:17 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:06:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertKeyExist (130.71s)
=== RUN   TestExipryGet
2021/12/23 11:06:22 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:06:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/23 11:06:29 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:06:34 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:06:39 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:06:44 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:06:44 App: TestExipryGet got deployed
2021/12/23 11:06:44 Starting bucket ops to source bucket
2021/12/23 11:07:29 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:07:34 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:07:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:07:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:07:34 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:07:39 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:07:44 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:07:49 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:07:54 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:07:54 App: TestExipryGet got un-deployed
2021/12/23 11:07:54 No eventing-con process running
2021/12/23 11:07:54 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/23 11:07:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:08:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:08:03 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:08:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:08:08 Pumping document in the destination bucket for expiry get
2021/12/23 11:08:08 Starting bucket ops to source bucket
2021/12/23 11:08:08 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:08:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/23 11:08:14 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:08:20 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:08:25 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:08:30 Waiting for app: TestExipryGet to get deployed
2021/12/23 11:08:30 App: TestExipryGet got deployed
2021/12/23 11:08:30 Starting bucket ops to source bucket
2021/12/23 11:09:15 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:09:20 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:09:25 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:09:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:09:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:09:25 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:09:30 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:09:35 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:09:40 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:09:45 Waiting for app: TestExipryGet to get un-deployed
2021/12/23 11:09:45 App: TestExipryGet got un-deployed
2021/12/23 11:09:45 No eventing-con process running
2021/12/23 11:09:45 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/23 11:09:47 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:09:57 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:09:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:09:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:10:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryGet (221.01s)
=== RUN   TestExipryInsert
2021/12/23 11:10:03 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:10:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/23 11:10:10 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:10:15 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:10:20 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:10:25 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:10:25 App: TestExipryInsert got deployed
2021/12/23 11:10:25 Starting bucket ops to source bucket
2021/12/23 11:11:10 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/23 11:11:15 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:11:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:11:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:11:15 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:11:20 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:11:25 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:11:30 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:11:35 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:11:35 App: TestExipryInsert got un-deployed
2021/12/23 11:11:35 No eventing-con process running
2021/12/23 11:11:35 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/23 11:11:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:11:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:11:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:11:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:11:49 Testing on the destination bucket for expiry insert
2021/12/23 11:11:49 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:11:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/23 11:11:55 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:12:00 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:12:05 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:12:10 Waiting for app: TestExipryInsert to get deployed
2021/12/23 11:12:10 App: TestExipryInsert got deployed
2021/12/23 11:12:10 Starting bucket ops to source bucket
2021/12/23 11:12:55 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:13:00 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:13:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:13:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:13:01 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:13:06 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:13:11 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:13:16 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:13:21 Waiting for app: TestExipryInsert to get un-deployed
2021/12/23 11:13:21 App: TestExipryInsert got un-deployed
2021/12/23 11:13:21 No eventing-con process running
2021/12/23 11:13:21 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/23 11:13:22 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:13:32 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:13:37 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:13:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:13:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:13:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryInsert (220.72s)
=== RUN   TestExipryReplace
2021/12/23 11:13:44 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:13:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/23 11:13:50 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:13:55 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:14:00 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:14:05 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:14:05 App: TestExipryReplace got deployed
2021/12/23 11:14:05 Starting bucket ops to source bucket
2021/12/23 11:14:51 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:14:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:14:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:14:51 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:14:56 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:15:01 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:15:06 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:15:11 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:15:11 App: TestExipryReplace got un-deployed
2021/12/23 11:15:11 No eventing-con process running
2021/12/23 11:15:11 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/23 11:15:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:15:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:15:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:15:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:15:25 Testing on the destination bucket for expiry insert
2021/12/23 11:15:25 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:15:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/23 11:15:31 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:15:36 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:15:41 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:15:46 Waiting for app: TestExipryReplace to get deployed
2021/12/23 11:15:46 App: TestExipryReplace got deployed
2021/12/23 11:15:46 Starting bucket ops to source bucket
2021/12/23 11:16:31 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:16:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:16:31 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:16:31 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:16:36 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:16:41 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:16:46 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:16:51 Waiting for app: TestExipryReplace to get un-deployed
2021/12/23 11:16:51 App: TestExipryReplace got un-deployed
2021/12/23 11:16:51 No eventing-con process running
2021/12/23 11:16:51 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/23 11:16:53 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:17:03 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:17:08 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:17:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:17:10 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:17:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryReplace (210.77s)
=== RUN   TestExipryUpsert
2021/12/23 11:17:15 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:17:16 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/23 11:17:21 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:17:26 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:17:31 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:17:36 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:17:36 App: TestExipryUpsert got deployed
2021/12/23 11:17:36 Starting bucket ops to source bucket
2021/12/23 11:18:21 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:18:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:18:21 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:18:21 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:18:26 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:18:31 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:18:36 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:18:41 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:18:41 App: TestExipryUpsert got un-deployed
2021/12/23 11:18:42 No eventing-con process running
2021/12/23 11:18:42 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/23 11:18:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:18:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:18:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:18:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:18:55 Testing on the destination bucket for expiry insert
2021/12/23 11:18:56 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:18:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/23 11:19:02 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:19:07 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:19:12 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:19:17 Waiting for app: TestExipryUpsert to get deployed
2021/12/23 11:19:17 App: TestExipryUpsert got deployed
2021/12/23 11:19:17 Starting bucket ops to source bucket
2021/12/23 11:20:02 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:20:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:20:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:20:02 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:20:07 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:20:12 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:20:17 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:20:22 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/23 11:20:22 App: TestExipryUpsert got un-deployed
2021/12/23 11:20:22 No eventing-con process running
2021/12/23 11:20:22 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/23 11:20:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:20:34 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:20:39 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:20:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:20:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:20:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryUpsert (210.65s)
=== RUN   TestCasReplace
2021/12/23 11:20:46 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:20:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/23 11:20:52 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:20:57 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:21:02 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:21:07 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:21:07 App: TestCasReplace got deployed
2021/12/23 11:21:07 Starting bucket ops to source bucket
2021/12/23 11:21:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:21:07 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:21:07 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:21:12 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:21:17 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:21:22 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:21:27 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:21:27 App: TestCasReplace got un-deployed
2021/12/23 11:21:27 No eventing-con process running
2021/12/23 11:21:27 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/23 11:21:29 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:21:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:21:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:21:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:21:41 Testing on the destination bucket for upsert with cas
2021/12/23 11:21:41 Starting bucket ops to source bucket
2021/12/23 11:21:41 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:21:42 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/23 11:21:47 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:21:52 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:21:57 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:22:02 Waiting for app: TestCasReplace to get deployed
2021/12/23 11:22:02 App: TestCasReplace got deployed
2021/12/23 11:22:02 Starting bucket ops to source bucket
2021/12/23 11:22:08 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:22:13 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:22:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:22:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:22:13 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:22:18 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:22:23 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:22:28 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:22:33 Waiting for app: TestCasReplace to get un-deployed
2021/12/23 11:22:33 App: TestCasReplace got un-deployed
2021/12/23 11:22:33 No eventing-con process running
2021/12/23 11:22:33 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/23 11:22:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:22:45 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:22:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:22:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:22:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasReplace (125.79s)
=== RUN   TestCasDelete
2021/12/23 11:22:51 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:22:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/23 11:22:57 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:23:03 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:23:08 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:23:13 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:23:13 App: TestCasDelete got deployed
2021/12/23 11:23:13 Starting bucket ops to source bucket
2021/12/23 11:23:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:23:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:23:13 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:23:18 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:23:23 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:23:28 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:23:33 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:23:33 App: TestCasDelete got un-deployed
2021/12/23 11:23:33 No eventing-con process running
2021/12/23 11:23:33 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/23 11:23:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:23:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:23:42 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:23:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:23:47 Testing on the destination bucket for delete with cas
2021/12/23 11:23:47 Starting bucket ops to source bucket
2021/12/23 11:23:47 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:23:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/23 11:23:53 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:23:58 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:24:03 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:24:08 Waiting for app: TestCasDelete to get deployed
2021/12/23 11:24:08 App: TestCasDelete got deployed
2021/12/23 11:24:08 Starting bucket ops to source bucket
2021/12/23 11:24:13 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:24:18 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:24:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:24:18 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:24:18 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 11:24:24 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:24:29 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:24:34 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:24:39 Waiting for app: TestCasDelete to get un-deployed
2021/12/23 11:24:39 App: TestCasDelete got un-deployed
2021/12/23 11:24:39 No eventing-con process running
2021/12/23 11:24:39 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/23 11:24:40 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:24:50 Waiting for dst bucket item count to get to: 0 curr count: 74
2021/12/23 11:24:55 Waiting for dst bucket item count to get to: 0 curr count: 74
2021/12/23 11:24:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:24:57 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:25:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasDelete (130.71s)
=== RUN   TestCountersIncrement
2021/12/23 11:25:02 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 11:25:08 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 11:25:09 Removing node(s): [] from the cluster
2021/12/23 11:25:14 Rebalance progress: 100
2021/12/23 11:25:14 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:25:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/23 11:25:20 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:25:25 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:25:30 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:25:35 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:25:35 App: TestCountersIncrement got deployed
2021/12/23 11:25:35 Starting bucket ops to source bucket
2021/12/23 11:25:41 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/23 11:25:46 Waiting for dst bucket item count to get to: 1024 curr count: 512
2021/12/23 11:25:46 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/23 11:25:46 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:25:51 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:25:56 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:26:01 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:26:06 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:26:06 App: TestCountersIncrement got un-deployed
2021/12/23 11:26:06 No eventing-con process running
2021/12/23 11:26:06 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/23 11:26:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:26:18 Waiting for dst bucket item count to get to: 0 curr count: 736
2021/12/23 11:26:23 Waiting for dst bucket item count to get to: 0 curr count: 223
2021/12/23 11:26:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:26:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:26:35 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 11:26:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:26:35 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:26:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/23 11:26:41 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:26:46 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:26:51 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:26:56 Waiting for app: TestCountersIncrement to get deployed
2021/12/23 11:26:56 App: TestCountersIncrement got deployed
2021/12/23 11:26:56 Starting bucket ops to source bucket
2021/12/23 11:27:02 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/23 11:27:07 Waiting for dst bucket item count to get to: 2048 curr count: 2013
2021/12/23 11:27:12 Waiting for dst bucket item count to get to: 2048 curr count: 2013
2021/12/23 11:27:12 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/23 11:27:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:27:17 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:27:23 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:27:28 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/23 11:27:28 App: TestCountersIncrement got un-deployed
2021/12/23 11:27:28 No eventing-con process running
2021/12/23 11:27:28 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/23 11:27:29 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:27:39 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 11:27:44 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/23 11:27:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:27:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:27:56 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/23 11:27:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:27:56 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/23 11:28:01 Rebalance progress: 100
--- PASS: TestCountersIncrement (179.16s)
=== RUN   TestCountersDecrement
2021/12/23 11:28:01 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 11:28:02 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 11:28:02 Removing node(s): [] from the cluster
2021/12/23 11:28:07 Rebalance progress: 100
2021/12/23 11:28:07 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:28:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/23 11:28:13 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:28:18 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:28:23 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:28:28 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:28:28 App: TestCountersDecrement got deployed
2021/12/23 11:28:28 Starting bucket ops to source bucket
2021/12/23 11:28:35 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/23 11:28:40 Waiting for dst bucket item count to get to: 1024 curr count: 368
2021/12/23 11:28:45 Waiting for dst bucket item count to get to: 1024 curr count: 368
2021/12/23 11:28:45 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/23 11:28:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:28:50 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:28:55 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:29:00 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:29:05 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:29:05 App: TestCountersDecrement got un-deployed
2021/12/23 11:29:05 No eventing-con process running
2021/12/23 11:29:05 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/23 11:29:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:29:17 Waiting for dst bucket item count to get to: 0 curr count: 1025
2021/12/23 11:29:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:29:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:29:28 Waiting for dst bucket item count to get to: 0 curr count: 869
2021/12/23 11:29:33 Waiting for dst bucket item count to get to: 0 curr count: 357
2021/12/23 11:29:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:29:33 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:29:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/23 11:29:40 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:29:45 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:29:50 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:29:55 Waiting for app: TestCountersDecrement to get deployed
2021/12/23 11:29:55 App: TestCountersDecrement got deployed
2021/12/23 11:29:55 Starting bucket ops to source bucket
2021/12/23 11:30:01 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/23 11:30:06 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/23 11:30:06 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/23 11:30:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:30:11 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:30:16 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:30:21 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:30:26 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/23 11:30:26 App: TestCountersDecrement got un-deployed
2021/12/23 11:30:26 No eventing-con process running
2021/12/23 11:30:26 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/23 11:30:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:30:38 Waiting for dst bucket item count to get to: 0 curr count: 1238
2021/12/23 11:30:43 Waiting for dst bucket item count to get to: 0 curr count: 214
2021/12/23 11:30:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:30:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:30:55 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/23 11:30:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:30:55 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/23 11:31:00 Rebalance progress: 100
--- PASS: TestCountersDecrement (178.54s)
=== RUN   TestCacheGet
2021/12/23 11:31:00 Deploying app: TestCacheGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:31:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet stored in temp store"
}
2021/12/23 11:31:06 Waiting for app: TestCacheGet to get deployed
2021/12/23 11:31:11 Waiting for app: TestCacheGet to get deployed
2021/12/23 11:31:16 Waiting for app: TestCacheGet to get deployed
2021/12/23 11:31:21 Waiting for app: TestCacheGet to get deployed
2021/12/23 11:31:21 App: TestCacheGet got deployed
2021/12/23 11:31:21 Starting bucket ops to source bucket
2021/12/23 11:31:26 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/23 11:31:31 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/23 11:31:36 Waiting for dst bucket item count to get to: 20 curr count: 5
2021/12/23 11:31:41 Waiting for dst bucket item count to get to: 20 curr count: 5
2021/12/23 11:31:46 Waiting for dst bucket item count to get to: 20 curr count: 18
2021/12/23 11:31:51 Waiting for dst bucket item count to get to: 20 curr count: 18
2021/12/23 11:31:51 src & dst bucket item count matched up. src bucket count: 20 dst bucket count: 20
2021/12/23 11:31:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:31:56 Waiting for app: TestCacheGet to get un-deployed
2021/12/23 11:32:01 Waiting for app: TestCacheGet to get un-deployed
2021/12/23 11:32:06 Waiting for app: TestCacheGet to get un-deployed
2021/12/23 11:32:11 Waiting for app: TestCacheGet to get un-deployed
2021/12/23 11:32:11 App: TestCacheGet got un-deployed
2021/12/23 11:32:11 No eventing-con process running
2021/12/23 11:32:11 Function: TestCacheGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet deleting in background"
}
2021/12/23 11:32:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:32:23 Waiting for dst bucket item count to get to: 0 curr count: 20
2021/12/23 11:32:28 Waiting for dst bucket item count to get to: 0 curr count: 10
2021/12/23 11:32:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:32:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:32:40 Waiting for dst bucket item count to get to: 0 curr count: 13
2021/12/23 11:32:45 Waiting for dst bucket item count to get to: 0 curr count: 13
2021/12/23 11:32:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheGet (105.20s)
=== RUN   TestCacheRYOW
2021/12/23 11:32:45 Deploying app: TestCacheRYOW. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:32:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW stored in temp store"
}
2021/12/23 11:32:51 Waiting for app: TestCacheRYOW to get deployed
2021/12/23 11:32:56 Waiting for app: TestCacheRYOW to get deployed
2021/12/23 11:33:01 Waiting for app: TestCacheRYOW to get deployed
2021/12/23 11:33:06 Waiting for app: TestCacheRYOW to get deployed
2021/12/23 11:33:06 App: TestCacheRYOW got deployed
2021/12/23 11:33:06 Starting bucket ops to source bucket
2021/12/23 11:33:11 Waiting for dst bucket item count to get to: 50 curr count: 0
2021/12/23 11:33:11 src & dst bucket item count matched up. src bucket count: 50 dst bucket count: 50
2021/12/23 11:33:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:33:17 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/23 11:33:22 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/23 11:33:27 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/23 11:33:32 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/23 11:33:32 App: TestCacheRYOW got un-deployed
2021/12/23 11:33:32 No eventing-con process running
2021/12/23 11:33:32 Function: TestCacheRYOW delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW deleting in background"
}
2021/12/23 11:33:33 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:33:43 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:33:48 Waiting for dst bucket item count to get to: 0 curr count: 25
2021/12/23 11:33:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:33:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:34:00 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/23 11:34:05 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/23 11:34:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheRYOW (80.26s)
=== RUN   TestCacheExpiry
2021/12/23 11:34:05 Deploying app: TestCacheExpiry. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:34:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry stored in temp store"
}
2021/12/23 11:34:11 Waiting for app: TestCacheExpiry to get deployed
2021/12/23 11:34:16 Waiting for app: TestCacheExpiry to get deployed
2021/12/23 11:34:21 Waiting for app: TestCacheExpiry to get deployed
2021/12/23 11:34:26 Waiting for app: TestCacheExpiry to get deployed
2021/12/23 11:34:26 App: TestCacheExpiry got deployed
2021/12/23 11:34:26 Starting bucket ops to source bucket
2021/12/23 11:34:32 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:34:37 Waiting for dst bucket item count to get to: 100 curr count: 19
2021/12/23 11:34:42 Waiting for dst bucket item count to get to: 100 curr count: 19
2021/12/23 11:34:47 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/23 11:34:52 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/23 11:34:52 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:34:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:34:57 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/23 11:35:02 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/23 11:35:07 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/23 11:35:12 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/23 11:35:12 App: TestCacheExpiry got un-deployed
2021/12/23 11:35:12 No eventing-con process running
2021/12/23 11:35:12 Function: TestCacheExpiry delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry deleting in background"
}
2021/12/23 11:35:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:35:24 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:35:29 Waiting for dst bucket item count to get to: 0 curr count: 25
2021/12/23 11:35:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:35:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:35:40 Waiting for dst bucket item count to get to: 0 curr count: 78
2021/12/23 11:35:45 Waiting for dst bucket item count to get to: 0 curr count: 78
2021/12/23 11:35:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiry (100.23s)
=== RUN   TestCacheExpiryCustom
2021/12/23 11:35:45 Deploying app: TestCacheExpiryCustom. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:35:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom stored in temp store"
}
2021/12/23 11:35:52 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/23 11:35:57 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/23 11:36:02 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/23 11:36:07 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/23 11:36:07 App: TestCacheExpiryCustom got deployed
2021/12/23 11:36:07 Starting bucket ops to source bucket
2021/12/23 11:36:12 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:36:17 Waiting for dst bucket item count to get to: 100 curr count: 19
2021/12/23 11:36:22 Waiting for dst bucket item count to get to: 100 curr count: 19
2021/12/23 11:36:27 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/23 11:36:32 Waiting for dst bucket item count to get to: 100 curr count: 94
2021/12/23 11:36:32 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:36:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:36:37 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/23 11:36:42 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/23 11:36:47 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/23 11:36:52 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/23 11:36:52 App: TestCacheExpiryCustom got un-deployed
2021/12/23 11:36:52 No eventing-con process running
2021/12/23 11:36:52 Function: TestCacheExpiryCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom deleting in background"
}
2021/12/23 11:36:54 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:37:04 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:37:09 Waiting for dst bucket item count to get to: 0 curr count: 25
2021/12/23 11:37:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:37:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:37:21 Waiting for dst bucket item count to get to: 0 curr count: 86
2021/12/23 11:37:26 Waiting for dst bucket item count to get to: 0 curr count: 86
2021/12/23 11:37:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiryCustom (100.25s)
=== RUN   TestCacheOverflowCustom
2021/12/23 11:37:26 Deploying app: TestCacheOverflowCustom. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:37:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom stored in temp store"
}
2021/12/23 11:37:32 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/23 11:37:37 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/23 11:37:42 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/23 11:37:47 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/23 11:37:47 App: TestCacheOverflowCustom got deployed
2021/12/23 11:37:47 Starting bucket ops to source bucket
2021/12/23 11:37:52 Waiting for dst bucket item count to get to: 35 curr count: 0
2021/12/23 11:37:57 Waiting for dst bucket item count to get to: 35 curr count: 6
2021/12/23 11:38:02 Waiting for dst bucket item count to get to: 35 curr count: 6
2021/12/23 11:38:07 Waiting for dst bucket item count to get to: 35 curr count: 26
2021/12/23 11:38:12 Waiting for dst bucket item count to get to: 35 curr count: 26
2021/12/23 11:38:12 src & dst bucket item count matched up. src bucket count: 35 dst bucket count: 35
2021/12/23 11:38:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:38:17 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/23 11:38:22 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/23 11:38:27 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/23 11:38:32 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/23 11:38:32 App: TestCacheOverflowCustom got un-deployed
2021/12/23 11:38:32 No eventing-con process running
2021/12/23 11:38:32 Function: TestCacheOverflowCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom deleting in background"
}
2021/12/23 11:38:34 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:38:44 Waiting for dst bucket item count to get to: 0 curr count: 5
2021/12/23 11:38:49 Waiting for dst bucket item count to get to: 0 curr count: 3
2021/12/23 11:38:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:38:51 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:39:01 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/23 11:39:06 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/23 11:39:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheOverflowCustom (100.33s)
=== RUN   TestStrictMode
2021/12/23 11:39:06 Deploying app: TestStrictMode. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:39:07 Context: Post to main store, response code: 422 dump: {
 "name": "ERR_HANDLER_COMPILATION",
 "code": 27,
 "description": {
  "area": "handlerCode",
  "column_number": 63,
  "compile_success": false,
  "description": "        destination[meta.id] = 00000000000000000000000000000101;\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nSyntaxError: Octal literals are not allowed in strict mode.\n",
  "index": 132,
  "language": "JavaScript",
  "line_number": 4
 }
}
--- PASS: TestStrictMode (1.06s)
=== RUN   TestDataTypes
2021/12/23 11:39:07 Deploying app: TestDataTypes. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:39:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes stored in temp store"
}
2021/12/23 11:39:13 Waiting for app: TestDataTypes to get deployed
2021/12/23 11:39:18 Waiting for app: TestDataTypes to get deployed
2021/12/23 11:39:23 Waiting for app: TestDataTypes to get deployed
2021/12/23 11:39:28 Waiting for app: TestDataTypes to get deployed
2021/12/23 11:39:28 App: TestDataTypes got deployed
2021/12/23 11:39:28 Starting bucket ops to source bucket
2021/12/23 11:39:33 Waiting for dst bucket item count to get to: 18 curr count: 0
2021/12/23 11:39:38 Waiting for dst bucket item count to get to: 18 curr count: 10
2021/12/23 11:39:43 Waiting for dst bucket item count to get to: 18 curr count: 10
2021/12/23 11:39:43 src & dst bucket item count matched up. src bucket count: 18 dst bucket count: 18
2021/12/23 11:39:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:39:49 Waiting for app: TestDataTypes to get un-deployed
2021/12/23 11:39:54 Waiting for app: TestDataTypes to get un-deployed
2021/12/23 11:39:59 Waiting for app: TestDataTypes to get un-deployed
2021/12/23 11:40:04 Waiting for app: TestDataTypes to get un-deployed
2021/12/23 11:40:04 App: TestDataTypes got un-deployed
2021/12/23 11:40:04 No eventing-con process running
2021/12/23 11:40:04 Function: TestDataTypes delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes deleting in background"
}
2021/12/23 11:40:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:40:15 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/23 11:40:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:40:17 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:40:27 Waiting for dst bucket item count to get to: 0 curr count: 18
2021/12/23 11:40:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDataTypes (80.19s)
=== RUN   TestEnoentGet
2021/12/23 11:40:27 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:40:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/23 11:40:33 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:40:38 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:40:43 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:40:48 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:40:48 App: TestEnoentGet got deployed
2021/12/23 11:40:48 Starting bucket ops to source bucket
2021/12/23 11:40:54 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:40:59 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/23 11:40:59 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:40:59 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:41:04 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:41:09 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:41:14 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:41:19 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:41:19 App: TestEnoentGet got un-deployed
2021/12/23 11:41:19 No eventing-con process running
2021/12/23 11:41:19 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/23 11:41:21 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:41:31 Waiting for dst bucket item count to get to: 0 curr count: 92
2021/12/23 11:41:36 Waiting for dst bucket item count to get to: 0 curr count: 92
2021/12/23 11:41:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:41:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:41:48 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 11:41:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:41:48 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:41:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/23 11:41:54 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:41:59 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:42:04 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:42:09 Waiting for app: TestEnoentGet to get deployed
2021/12/23 11:42:09 App: TestEnoentGet got deployed
2021/12/23 11:42:09 Starting bucket ops to source bucket
2021/12/23 11:42:14 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:42:19 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/23 11:42:19 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:42:19 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:42:24 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:42:29 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:42:34 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:42:39 Waiting for app: TestEnoentGet to get un-deployed
2021/12/23 11:42:39 App: TestEnoentGet got un-deployed
2021/12/23 11:42:39 No eventing-con process running
2021/12/23 11:42:39 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/23 11:42:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:42:51 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:42:56 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:42:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:42:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:43:08 Waiting for dst bucket item count to get to: 0 curr count: 65
2021/12/23 11:43:13 Waiting for dst bucket item count to get to: 0 curr count: 15
2021/12/23 11:43:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentGet (165.70s)
=== RUN   TestEnoentDelete
2021/12/23 11:43:13 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:43:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/23 11:43:19 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:43:24 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:43:29 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:43:34 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:43:34 App: TestEnoentDelete got deployed
2021/12/23 11:43:34 Starting bucket ops to source bucket
2021/12/23 11:43:39 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:43:44 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/23 11:43:44 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:43:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:43:50 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:43:55 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:44:00 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:44:05 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:44:05 App: TestEnoentDelete got un-deployed
2021/12/23 11:44:05 No eventing-con process running
2021/12/23 11:44:05 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/23 11:44:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:44:16 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:44:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:44:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:44:28 Waiting for dst bucket item count to get to: 0 curr count: 75
2021/12/23 11:44:33 Waiting for dst bucket item count to get to: 0 curr count: 25
2021/12/23 11:44:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:44:33 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:44:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/23 11:44:39 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:44:44 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:44:49 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:44:54 Waiting for app: TestEnoentDelete to get deployed
2021/12/23 11:44:54 App: TestEnoentDelete got deployed
2021/12/23 11:44:54 Starting bucket ops to source bucket
2021/12/23 11:45:00 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:45:05 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/23 11:45:05 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:45:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:45:10 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:45:15 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:45:20 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/23 11:45:20 App: TestEnoentDelete got un-deployed
2021/12/23 11:45:20 No eventing-con process running
2021/12/23 11:45:20 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/23 11:45:22 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:45:32 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:45:37 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:45:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:45:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:45:49 Waiting for dst bucket item count to get to: 0 curr count: 83
2021/12/23 11:45:54 Waiting for dst bucket item count to get to: 0 curr count: 33
2021/12/23 11:45:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentDelete (160.61s)
=== RUN   TestError
2021/12/23 11:46:04 Waiting for app: TestError to get un-deployed
2021/12/23 11:46:04 App: TestError got un-deployed
2021/12/23 11:46:04 No eventing-con process running
2021/12/23 11:46:04 Function: TestError delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestError not found"
}
2021/12/23 11:46:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:46:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:46:12 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:46:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:46:17 Deploying app: TestError. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:46:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError stored in temp store"
}
2021/12/23 11:46:23 Waiting for app: TestError to get deployed
2021/12/23 11:46:28 Waiting for app: TestError to get deployed
2021/12/23 11:46:33 Waiting for app: TestError to get deployed
2021/12/23 11:46:38 Waiting for app: TestError to get deployed
2021/12/23 11:46:38 App: TestError got deployed
2021/12/23 11:46:38 Starting bucket ops to source bucket
2021/12/23 11:46:44 Waiting for dst bucket item count to get to: 300 curr count: 0
2021/12/23 11:46:49 Waiting for dst bucket item count to get to: 300 curr count: 144
2021/12/23 11:46:49 src & dst bucket item count matched up. src bucket count: 300 dst bucket count: 300
2021/12/23 11:46:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:46:54 Waiting for app: TestError to get un-deployed
2021/12/23 11:46:59 Waiting for app: TestError to get un-deployed
2021/12/23 11:47:04 Waiting for app: TestError to get un-deployed
2021/12/23 11:47:09 Waiting for app: TestError to get un-deployed
2021/12/23 11:47:09 App: TestError got un-deployed
2021/12/23 11:47:09 No eventing-con process running
2021/12/23 11:47:09 Function: TestError delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError deleting in background"
}
2021/12/23 11:47:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:47:21 Waiting for dst bucket item count to get to: 0 curr count: 84
2021/12/23 11:47:26 Waiting for dst bucket item count to get to: 0 curr count: 84
2021/12/23 11:47:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:47:27 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:47:37 Waiting for dst bucket item count to get to: 0 curr count: 144
2021/12/23 11:47:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestError (103.86s)
=== RUN   TestCRLF
2021/12/23 11:47:47 Waiting for app: TestCRLF to get un-deployed
2021/12/23 11:47:47 App: TestCRLF got un-deployed
2021/12/23 11:47:47 No eventing-con process running
2021/12/23 11:47:47 Function: TestCRLF delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCRLF not found"
}
2021/12/23 11:47:49 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:47:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:47:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:48:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:48:01 Deploying app: TestCRLF. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:48:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF stored in temp store"
}
2021/12/23 11:48:07 Waiting for app: TestCRLF to get deployed
2021/12/23 11:48:12 Waiting for app: TestCRLF to get deployed
2021/12/23 11:48:17 Waiting for app: TestCRLF to get deployed
2021/12/23 11:48:22 Waiting for app: TestCRLF to get deployed
2021/12/23 11:48:22 App: TestCRLF got deployed
2021/12/23 11:48:22 Starting bucket ops to source bucket
2021/12/23 11:48:27 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:48:32 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 11:48:32 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 11:48:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:48:38 Waiting for app: TestCRLF to get un-deployed
2021/12/23 11:48:43 Waiting for app: TestCRLF to get un-deployed
2021/12/23 11:48:48 Waiting for app: TestCRLF to get un-deployed
2021/12/23 11:48:53 Waiting for app: TestCRLF to get un-deployed
2021/12/23 11:48:53 App: TestCRLF got un-deployed
2021/12/23 11:48:53 No eventing-con process running
2021/12/23 11:48:53 Function: TestCRLF delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF deleting in background"
}
2021/12/23 11:48:54 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:04 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:49:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:49:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:16 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 11:49:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCRLF (98.77s)
=== RUN   TestImportExport
2021/12/23 11:49:26 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:49:26 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/23 11:49:26 No eventing-con process running
2021/12/23 11:49:26 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_n1ql_insert_on_update not found"
}
2021/12/23 11:49:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:49:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:49:45 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:49:45 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/23 11:49:45 No eventing-con process running
2021/12/23 11:49:45 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_bucket_op_on_update not found"
}
2021/12/23 11:49:47 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:49:53 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:49:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:49:58 Deploying app: TestImportExport_n1ql_insert_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:50:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update stored in temp store"
}
2021/12/23 11:50:00 Deploying app: TestImportExport_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:50:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update stored in temp store"
}
2021/12/23 11:50:06 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/23 11:50:11 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/23 11:50:16 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/23 11:50:21 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/23 11:50:21 App: TestImportExport_n1ql_insert_on_update got deployed
2021/12/23 11:50:26 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/23 11:50:31 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/23 11:50:36 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/23 11:50:36 App: TestImportExport_bucket_op_on_update got deployed
2021/12/23 11:50:41 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:50:46 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:50:51 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:50:56 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:50:56 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/23 11:50:56 No eventing-con process running
2021/12/23 11:50:56 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/23 11:51:01 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:51:06 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:51:11 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:51:16 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:51:16 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/23 11:51:16 No eventing-con process running
2021/12/23 11:51:16 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/23 11:51:34 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/23 11:51:34 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/23 11:51:34 No eventing-con process running
2021/12/23 11:51:34 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/23 11:51:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:51:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:51:42 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:51:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:51:52 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/23 11:51:52 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/23 11:51:52 No eventing-con process running
2021/12/23 11:51:52 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/23 11:51:54 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:51:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:52:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:52:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestImportExport (169.73s)
=== RUN   TestDeployUndeployLoopNonDefaultSettings
2021/12/23 11:52:16 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:52:16 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:52:16 No eventing-con process running
2021/12/23 11:52:16 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopNonDefaultSettings not found"
}
2021/12/23 11:52:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:52:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:52:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:52:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:52:29 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:52:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/23 11:52:31 Starting bucket ops to source bucket
2021/12/23 11:52:39 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:52:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:52:49 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:52:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:52:54 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 11:52:59 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/23 11:52:59 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/23 11:52:59 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:52:59 Undeploying app: bucket_op_on_update
2021/12/23 11:53:04 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:53:09 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:53:14 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:53:19 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:53:19 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:53:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:53:19 No eventing-con process running
2021/12/23 11:53:21 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:53:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:54:03 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:54:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/23 11:54:04 Starting bucket ops to source bucket
2021/12/23 11:54:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:54:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:54:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:54:27 Waiting for dst bucket item count to get to: 5000 curr count: 1642
2021/12/23 11:54:32 Waiting for dst bucket item count to get to: 5000 curr count: 4142
2021/12/23 11:54:32 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 11:54:37 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/23 11:54:37 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/23 11:54:37 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:54:37 Undeploying app: bucket_op_on_update
2021/12/23 11:54:42 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:54:47 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:54:52 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:54:57 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:54:57 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:54:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:54:57 No eventing-con process running
2021/12/23 11:54:59 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:55:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:55:41 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:55:43 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/23 11:55:43 Starting bucket ops to source bucket
2021/12/23 11:55:50 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:55:55 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:56:00 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:56:06 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:56:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 11:56:11 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/23 11:56:11 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/23 11:56:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:56:11 Undeploying app: bucket_op_on_update
2021/12/23 11:56:16 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:56:21 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:56:26 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:56:31 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:56:31 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:56:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:56:31 No eventing-con process running
2021/12/23 11:56:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:56:40 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:57:15 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:57:16 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/23 11:57:16 Starting bucket ops to source bucket
2021/12/23 11:57:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:57:29 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:57:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:57:39 Waiting for dst bucket item count to get to: 5000 curr count: 2357
2021/12/23 11:57:44 Waiting for dst bucket item count to get to: 5000 curr count: 2357
2021/12/23 11:57:44 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 11:57:49 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/23 11:57:49 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/23 11:57:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:57:49 Undeploying app: bucket_op_on_update
2021/12/23 11:57:54 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:57:59 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:58:04 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:58:09 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:58:09 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:58:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:58:09 No eventing-con process running
2021/12/23 11:58:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:58:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:58:53 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 11:58:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/23 11:58:54 Starting bucket ops to source bucket
2021/12/23 11:59:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:59:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:59:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 11:59:17 Waiting for dst bucket item count to get to: 5000 curr count: 1331
2021/12/23 11:59:22 Waiting for dst bucket item count to get to: 5000 curr count: 3831
2021/12/23 11:59:22 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 11:59:27 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/23 11:59:27 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/23 11:59:27 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 11:59:27 Undeploying app: bucket_op_on_update
2021/12/23 11:59:32 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:59:37 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:59:43 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:59:48 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/23 11:59:48 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/23 11:59:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 11:59:48 No eventing-con process running
2021/12/23 11:59:49 Context: Bucket flush, response code: 200 dump: 
2021/12/23 11:59:57 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:00:32 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings deleting in background"
}
--- PASS: TestDeployUndeployLoopNonDefaultSettings (505.67s)
=== RUN   TestOnUpdateN1QLOp
2021/12/23 12:00:42 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/23 12:00:42 App: TestOnUpdateN1QLOp got un-deployed
2021/12/23 12:00:42 No eventing-con process running
2021/12/23 12:00:42 Function: TestOnUpdateN1QLOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateN1QLOp not found"
}
2021/12/23 12:00:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:00:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:00:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:00:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:00:55 Deploying app: TestOnUpdateN1QLOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:00:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp stored in temp store"
}
2021/12/23 12:00:56 Starting bucket ops to source bucket
2021/12/23 12:01:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:01:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:01:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:01:19 Waiting for dst bucket item count to get to: 5000 curr count: 551
2021/12/23 12:01:24 Waiting for dst bucket item count to get to: 5000 curr count: 551
2021/12/23 12:01:24 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:01:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:01:29 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/23 12:01:35 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/23 12:01:40 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/23 12:01:45 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/23 12:01:45 App: TestOnUpdateN1QLOp got un-deployed
2021/12/23 12:01:45 No eventing-con process running
2021/12/23 12:01:45 Function: TestOnUpdateN1QLOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp deleting in background"
}
2021/12/23 12:01:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:01:56 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:01:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:01:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:02:08 Waiting for dst bucket item count to get to: 0 curr count: 4053
2021/12/23 12:02:13 Waiting for dst bucket item count to get to: 0 curr count: 1464
2021/12/23 12:02:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateN1QLOp (101.66s)
=== RUN   TestOnUpdateBucketOpDefaultSettings
2021/12/23 12:02:23 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/23 12:02:23 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/23 12:02:23 No eventing-con process running
2021/12/23 12:02:23 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings not found"
}
2021/12/23 12:02:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:02:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:02:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:02:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:02:37 Deploying app: TestOnUpdateBucketOpDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:02:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings stored in temp store"
}
2021/12/23 12:02:38 Starting bucket ops to source bucket
2021/12/23 12:02:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:02:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:02:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:03:01 Waiting for dst bucket item count to get to: 5000 curr count: 289
2021/12/23 12:03:06 Waiting for dst bucket item count to get to: 5000 curr count: 289
2021/12/23 12:03:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:03:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:03:11 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/23 12:03:16 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/23 12:03:21 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/23 12:03:26 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/23 12:03:26 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/23 12:03:26 No eventing-con process running
2021/12/23 12:03:26 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings deleting in background"
}
2021/12/23 12:03:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:03:38 Waiting for dst bucket item count to get to: 0 curr count: 3185
2021/12/23 12:03:43 Waiting for dst bucket item count to get to: 0 curr count: 685
2021/12/23 12:03:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:03:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:03:55 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:03:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings (101.54s)
=== RUN   TestOnUpdateBucketOpNonDefaultSettings
2021/12/23 12:04:05 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 12:04:05 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/23 12:04:05 No eventing-con process running
2021/12/23 12:04:05 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/23 12:04:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:04:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:04:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:04:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:04:18 Deploying app: TestOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:04:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/23 12:04:20 Starting bucket ops to source bucket
2021/12/23 12:04:27 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:04:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:04:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:04:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:04:47 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/23 12:04:47 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:04:47 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:04:53 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 12:04:58 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 12:05:03 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 12:05:08 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 12:05:08 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/23 12:05:08 No eventing-con process running
2021/12/23 12:05:08 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/23 12:05:10 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:05:20 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 12:05:25 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 12:05:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:05:26 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:05:36 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:05:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpNonDefaultSettings (101.59s)
=== RUN   TestOnUpdateBucketOpDefaultSettings10K
2021/12/23 12:05:46 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/23 12:05:46 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/23 12:05:46 No eventing-con process running
2021/12/23 12:05:46 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings10K not found"
}
2021/12/23 12:05:48 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:05:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:05:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:06:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:06:00 Deploying app: TestOnUpdateBucketOpDefaultSettings10K. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:06:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K stored in temp store"
}
2021/12/23 12:06:01 Starting bucket ops to source bucket
2021/12/23 12:06:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:06:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:06:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:06:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:06:24 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:06:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:06:29 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/23 12:06:34 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/23 12:06:39 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/23 12:06:44 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/23 12:06:44 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/23 12:06:44 No eventing-con process running
2021/12/23 12:06:44 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K deleting in background"
}
2021/12/23 12:06:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:06:56 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:06:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:06:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:07:08 Waiting for dst bucket item count to get to: 0 curr count: 3347
2021/12/23 12:07:13 Waiting for dst bucket item count to get to: 0 curr count: 847
2021/12/23 12:07:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings10K (96.58s)
=== RUN   TestOnUpdateBucketOpDefaultSettings100K
2021/12/23 12:07:23 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/23 12:07:23 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/23 12:07:23 No eventing-con process running
2021/12/23 12:07:23 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings100K not found"
}
2021/12/23 12:07:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:07:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:07:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:07:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:07:37 Deploying app: TestOnUpdateBucketOpDefaultSettings100K. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:07:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K stored in temp store"
}
2021/12/23 12:07:38 Starting bucket ops to source bucket
2021/12/23 12:07:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:07:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:07:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:08:01 Waiting for dst bucket item count to get to: 5000 curr count: 27
2021/12/23 12:08:06 Waiting for dst bucket item count to get to: 5000 curr count: 27
2021/12/23 12:08:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:08:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:08:11 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/23 12:08:16 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/23 12:08:21 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/23 12:08:26 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/23 12:08:26 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/23 12:08:26 No eventing-con process running
2021/12/23 12:08:26 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K deleting in background"
}
2021/12/23 12:08:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:08:38 Waiting for dst bucket item count to get to: 0 curr count: 3566
2021/12/23 12:08:43 Waiting for dst bucket item count to get to: 0 curr count: 1066
2021/12/23 12:08:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:08:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:08:55 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:08:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings100K (102.07s)
=== RUN   TestOnDeleteBucketOp
2021/12/23 12:09:05 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/23 12:09:05 App: TestOnDeleteBucketOp got un-deployed
2021/12/23 12:09:05 No eventing-con process running
2021/12/23 12:09:05 Function: TestOnDeleteBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp not found"
}
2021/12/23 12:09:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:09:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:09:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:09:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:09:19 Deploying app: TestOnDeleteBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:09:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp stored in temp store"
}
2021/12/23 12:09:20 Starting bucket ops to source bucket
2021/12/23 12:09:31 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:09:36 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:09:41 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:09:46 Waiting for dst bucket item count to get to: 5000 curr count: 2504
2021/12/23 12:09:46 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:09:46 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:09:51 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/23 12:09:56 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/23 12:10:01 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/23 12:10:06 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/23 12:10:06 App: TestOnDeleteBucketOp got un-deployed
2021/12/23 12:10:06 No eventing-con process running
2021/12/23 12:10:06 Function: TestOnDeleteBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp deleting in background"
}
2021/12/23 12:10:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:10:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:10:15 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:10:25 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:10:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp (89.94s)
=== RUN   TestOnDeleteBucketOp5K
2021/12/23 12:10:35 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/23 12:10:35 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/23 12:10:35 No eventing-con process running
2021/12/23 12:10:35 Function: TestOnDeleteBucketOp5K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp5K not found"
}
2021/12/23 12:10:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:10:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:10:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:10:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:10:49 Deploying app: TestOnDeleteBucketOp5K. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:10:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K stored in temp store"
}
2021/12/23 12:10:50 Starting bucket ops to source bucket
2021/12/23 12:11:01 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:11:06 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:11:11 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:11:16 Waiting for dst bucket item count to get to: 5000 curr count: 2504
2021/12/23 12:11:16 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:11:16 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:11:21 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/23 12:11:26 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/23 12:11:31 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/23 12:11:36 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/23 12:11:36 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/23 12:11:36 No eventing-con process running
2021/12/23 12:11:36 Function: TestOnDeleteBucketOp5K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K deleting in background"
}
2021/12/23 12:11:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:11:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:11:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:11:55 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:11:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp5K (89.95s)
=== RUN   TestTimerBucketOp
2021/12/23 12:12:05 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/23 12:12:05 App: TestTimerBucketOp got un-deployed
2021/12/23 12:12:05 No eventing-con process running
2021/12/23 12:12:05 Function: TestTimerBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerBucketOp not found"
}
2021/12/23 12:12:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:12:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:12:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:12:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:12:18 Deploying app: TestTimerBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:12:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp stored in temp store"
}
2021/12/23 12:12:20 Starting bucket ops to source bucket
2021/12/23 12:12:27 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:38 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:48 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:12:58 Waiting for dst bucket item count to get to: 5000 curr count: 1251
2021/12/23 12:13:03 Waiting for dst bucket item count to get to: 5000 curr count: 3751
2021/12/23 12:13:03 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:13:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:13:08 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/23 12:13:13 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/23 12:13:18 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/23 12:13:23 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/23 12:13:23 App: TestTimerBucketOp got un-deployed
2021/12/23 12:13:23 No eventing-con process running
2021/12/23 12:13:23 Function: TestTimerBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp deleting in background"
}
2021/12/23 12:13:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:13:35 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:13:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:13:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:13:46 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:13:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerBucketOp (111.46s)
=== RUN   TestTimerInPastBucketOp
2021/12/23 12:13:56 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/23 12:13:56 App: TestTimerInPastBucketOp got un-deployed
2021/12/23 12:13:56 No eventing-con process running
2021/12/23 12:13:56 Function: TestTimerInPastBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerInPastBucketOp not found"
}
2021/12/23 12:13:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:14:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:14:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:14:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:14:10 Deploying app: TestTimerInPastBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:14:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp stored in temp store"
}
2021/12/23 12:14:11 Starting bucket ops to source bucket
2021/12/23 12:14:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:29 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:39 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:14:49 Waiting for dst bucket item count to get to: 5000 curr count: 2126
2021/12/23 12:14:54 Waiting for dst bucket item count to get to: 5000 curr count: 2126
2021/12/23 12:14:54 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:14:54 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:14:59 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/23 12:15:04 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/23 12:15:09 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/23 12:15:14 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/23 12:15:14 App: TestTimerInPastBucketOp got un-deployed
2021/12/23 12:15:14 No eventing-con process running
2021/12/23 12:15:14 Function: TestTimerInPastBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp deleting in background"
}
2021/12/23 12:15:16 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:15:26 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:15:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:15:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:15:38 Waiting for dst bucket item count to get to: 0 curr count: 3582
2021/12/23 12:15:43 Waiting for dst bucket item count to get to: 0 curr count: 1082
2021/12/23 12:15:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerInPastBucketOp (116.70s)
=== RUN   TestDeployUndeployLoopTimer
2021/12/23 12:15:53 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:15:53 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:15:53 No eventing-con process running
2021/12/23 12:15:53 Function: TestDeployUndeployLoopTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopTimer not found"
}
2021/12/23 12:15:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:16:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:16:02 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:16:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:16:07 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:16:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/23 12:16:08 Starting bucket ops to source bucket
2021/12/23 12:16:16 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:21 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:26 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:31 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:36 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:41 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:16:46 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:16:51 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/23 12:16:51 App: TestDeployUndeployLoopTimer got deployed
2021/12/23 12:16:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:16:51 Undeploying app: bucket_op_with_timer
2021/12/23 12:16:56 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:17:01 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:17:06 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:17:11 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:17:11 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:17:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:17:11 No eventing-con process running
2021/12/23 12:17:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:17:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:17:55 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:17:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/23 12:17:56 Starting bucket ops to source bucket
2021/12/23 12:18:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:29 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:18:34 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:18:39 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/23 12:18:39 App: TestDeployUndeployLoopTimer got deployed
2021/12/23 12:18:39 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:18:39 Undeploying app: bucket_op_with_timer
2021/12/23 12:18:44 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:18:49 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:18:54 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:18:59 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:18:59 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:18:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:18:59 No eventing-con process running
2021/12/23 12:19:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:19:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:19:43 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:19:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/23 12:19:44 Starting bucket ops to source bucket
2021/12/23 12:19:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:19:57 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:20:27 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/23 12:20:27 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:20:32 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/23 12:20:32 App: TestDeployUndeployLoopTimer got deployed
2021/12/23 12:20:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:20:32 Undeploying app: bucket_op_with_timer
2021/12/23 12:20:37 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:20:42 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:20:47 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:20:52 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:20:52 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:20:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:20:53 No eventing-con process running
2021/12/23 12:20:54 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:21:02 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:21:37 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:21:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/23 12:21:38 Starting bucket ops to source bucket
2021/12/23 12:21:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:21:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:21:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:22:01 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:22:06 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:22:11 Waiting for dst bucket item count to get to: 5000 curr count: 992
2021/12/23 12:22:16 Waiting for dst bucket item count to get to: 5000 curr count: 3335
2021/12/23 12:22:16 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:22:21 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/23 12:22:21 App: TestDeployUndeployLoopTimer got deployed
2021/12/23 12:22:21 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:22:21 Undeploying app: bucket_op_with_timer
2021/12/23 12:22:26 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:22:31 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:22:36 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:22:41 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:22:41 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:22:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:22:41 No eventing-con process running
2021/12/23 12:22:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:22:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:23:25 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:23:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/23 12:23:26 Starting bucket ops to source bucket
2021/12/23 12:23:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:23:39 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:23:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:23:49 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:23:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:23:59 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:24:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:24:04 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:24:09 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/23 12:24:09 App: TestDeployUndeployLoopTimer got deployed
2021/12/23 12:24:09 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:24:09 Undeploying app: bucket_op_with_timer
2021/12/23 12:24:14 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:24:19 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:24:24 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:24:29 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/23 12:24:29 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/23 12:24:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:24:29 No eventing-con process running
2021/12/23 12:24:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:24:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:25:13 Function: TestDeployUndeployLoopTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer deleting in background"
}
--- PASS: TestDeployUndeployLoopTimer (570.21s)
=== RUN   TestMultipleHandlers
2021/12/23 12:25:23 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/23 12:25:23 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/23 12:25:23 No eventing-con process running
2021/12/23 12:25:23 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_update not found"
}
2021/12/23 12:25:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:25:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:25:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:25:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:25:42 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/23 12:25:42 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/23 12:25:42 No eventing-con process running
2021/12/23 12:25:42 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_delete not found"
}
2021/12/23 12:25:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:25:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:25:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:25:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:25:56 Deploying app: TestMultipleHandlers_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:25:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update stored in temp store"
}
2021/12/23 12:25:57 Deploying app: TestMultipleHandlers_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:25:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete stored in temp store"
}
2021/12/23 12:26:03 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/23 12:26:08 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/23 12:26:13 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/23 12:26:18 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/23 12:26:18 App: TestMultipleHandlers_bucket_op_on_update got deployed
2021/12/23 12:26:23 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/23 12:26:28 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/23 12:26:33 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/23 12:26:33 App: TestMultipleHandlers_bucket_op_on_delete got deployed
2021/12/23 12:26:33 Starting bucket ops to source bucket
2021/12/23 12:26:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:26:48 Waiting for dst bucket item count to get to: 5000 curr count: 4524
2021/12/23 12:26:53 Waiting for dst bucket item count to get to: 5000 curr count: 4524
2021/12/23 12:26:53 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:26:53 Starting bucket ops to source bucket
2021/12/23 12:27:08 Waiting for dst bucket item count to get to: 10000 curr count: 5658
2021/12/23 12:27:13 Waiting for dst bucket item count to get to: 10000 curr count: 5658
2021/12/23 12:27:13 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 12:27:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:27:13 Function: TestMultipleHandlers_bucket_op_on_update update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:27:13 Function: TestMultipleHandlers_bucket_op_on_delete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:27:18 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/23 12:27:23 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/23 12:27:28 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/23 12:27:33 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/23 12:27:33 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/23 12:27:33 No eventing-con process running
2021/12/23 12:27:33 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update deleting in background"
}
2021/12/23 12:27:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:27:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:27:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:27:53 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 12:27:58 Waiting for dst bucket item count to get to: 0 curr count: 4996
2021/12/23 12:27:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:28:03 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/23 12:28:08 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/23 12:28:13 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/23 12:28:18 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/23 12:28:18 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/23 12:28:18 No eventing-con process running
2021/12/23 12:28:18 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete deleting in background"
}
2021/12/23 12:28:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:28:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:28:26 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:28:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMultipleHandlers (198.09s)
=== RUN   TestPauseResumeLoopDefaultSettings
2021/12/23 12:28:41 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/23 12:28:41 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/23 12:28:41 No eventing-con process running
2021/12/23 12:28:41 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopDefaultSettings not found"
}
2021/12/23 12:28:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:28:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:28:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:28:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:28:55 Deploying app: TestPauseResumeLoopDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:28:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings stored in temp store"
}
2021/12/23 12:28:56 Starting bucket ops to source bucket
2021/12/23 12:29:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:29:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:29:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:29:19 Waiting for dst bucket item count to get to: 5000 curr count: 2431
2021/12/23 12:29:24 Waiting for dst bucket item count to get to: 5000 curr count: 2431
2021/12/23 12:29:24 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:29:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:29:29 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/23 12:29:29 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/23 12:29:29 Pausing app: bucket_op_on_update
2021/12/23 12:29:29 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:29:34 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/23 12:29:34 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/23 12:29:34 Resuming app: bucket_op_on_update
2021/12/23 12:29:35 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:29:35 Starting bucket ops to source bucket
2021/12/23 12:29:43 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:29:48 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:29:53 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:29:53 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 12:29:53 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:29:58 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/23 12:29:58 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/23 12:29:58 Pausing app: bucket_op_on_update
2021/12/23 12:29:58 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:30:03 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/23 12:30:03 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/23 12:30:03 Resuming app: bucket_op_on_update
2021/12/23 12:30:05 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:30:05 Starting bucket ops to source bucket
2021/12/23 12:30:13 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:30:18 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:30:23 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:30:23 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/23 12:30:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:30:28 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/23 12:30:28 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/23 12:30:28 Pausing app: bucket_op_on_update
2021/12/23 12:30:28 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:30:33 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/23 12:30:33 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/23 12:30:33 Resuming app: bucket_op_on_update
2021/12/23 12:30:34 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:30:34 Starting bucket ops to source bucket
2021/12/23 12:30:42 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:30:47 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:30:52 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:30:52 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/23 12:30:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:30:57 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/23 12:30:57 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/23 12:30:57 Pausing app: bucket_op_on_update
2021/12/23 12:30:57 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:31:02 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/23 12:31:02 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/23 12:31:02 Resuming app: bucket_op_on_update
2021/12/23 12:31:03 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:31:03 Starting bucket ops to source bucket
2021/12/23 12:31:11 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:31:16 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:31:21 Waiting for dst bucket item count to get to: 25000 curr count: 20027
2021/12/23 12:31:26 Waiting for dst bucket item count to get to: 25000 curr count: 20027
2021/12/23 12:31:26 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/23 12:31:26 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:31:31 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/23 12:31:31 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/23 12:31:31 Pausing app: bucket_op_on_update
2021/12/23 12:31:31 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:31:36 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/23 12:31:36 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/23 12:31:41 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/23 12:31:46 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/23 12:31:51 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/23 12:31:56 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/23 12:31:57 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/23 12:31:57 No eventing-con process running
2021/12/23 12:31:57 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings deleting in background"
}
2021/12/23 12:31:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:32:08 Waiting for dst bucket item count to get to: 0 curr count: 19860
2021/12/23 12:32:13 Waiting for dst bucket item count to get to: 0 curr count: 7360
2021/12/23 12:32:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:32:15 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:32:25 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/23 12:32:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopDefaultSettings (233.77s)
=== RUN   TestPauseResumeLoopNonDefaultSettings
2021/12/23 12:32:35 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/23 12:32:35 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/23 12:32:35 No eventing-con process running
2021/12/23 12:32:35 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopNonDefaultSettings not found"
}
2021/12/23 12:32:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:32:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:32:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:32:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:32:49 Deploying app: TestPauseResumeLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:32:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings stored in temp store"
}
2021/12/23 12:32:50 Starting bucket ops to source bucket
2021/12/23 12:32:58 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:33:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:33:08 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:33:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:33:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:33:23 Waiting for dst bucket item count to get to: 5000 curr count: 185
2021/12/23 12:33:28 Waiting for dst bucket item count to get to: 5000 curr count: 2381
2021/12/23 12:33:33 Waiting for dst bucket item count to get to: 5000 curr count: 4696
2021/12/23 12:33:33 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:33:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:33:38 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/23 12:33:38 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/23 12:33:38 Pausing app: bucket_op_with_timer
2021/12/23 12:33:38 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:33:43 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/23 12:33:43 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/23 12:33:43 Resuming app: bucket_op_with_timer
2021/12/23 12:33:44 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:33:44 Starting bucket ops to source bucket
2021/12/23 12:33:52 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:33:57 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:34:02 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:34:07 Waiting for dst bucket item count to get to: 10000 curr count: 5890
2021/12/23 12:34:12 Waiting for dst bucket item count to get to: 10000 curr count: 5890
2021/12/23 12:34:12 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 12:34:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:34:17 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/23 12:34:17 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/23 12:34:17 Pausing app: bucket_op_with_timer
2021/12/23 12:34:17 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:34:22 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/23 12:34:22 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/23 12:34:22 Resuming app: bucket_op_with_timer
2021/12/23 12:34:23 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:34:23 Starting bucket ops to source bucket
2021/12/23 12:34:31 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:34:36 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:34:41 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:34:46 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:34:51 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/23 12:34:51 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/23 12:34:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:34:56 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/23 12:34:56 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/23 12:34:56 Pausing app: bucket_op_with_timer
2021/12/23 12:34:57 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:35:02 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/23 12:35:02 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/23 12:35:02 Resuming app: bucket_op_with_timer
2021/12/23 12:35:03 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:35:03 Starting bucket ops to source bucket
2021/12/23 12:35:11 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:16 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:21 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:26 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:31 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:36 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/23 12:35:36 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/23 12:35:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:35:41 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/23 12:35:41 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/23 12:35:41 Pausing app: bucket_op_with_timer
2021/12/23 12:35:41 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:35:46 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/23 12:35:46 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/23 12:35:46 Resuming app: bucket_op_with_timer
2021/12/23 12:35:47 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:35:47 Starting bucket ops to source bucket
2021/12/23 12:35:55 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:36:00 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:36:05 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:36:10 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:36:15 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/23 12:36:20 Waiting for dst bucket item count to get to: 25000 curr count: 21204
2021/12/23 12:36:25 Waiting for dst bucket item count to get to: 25000 curr count: 21204
2021/12/23 12:36:25 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/23 12:36:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:36:30 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/23 12:36:30 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/23 12:36:30 Pausing app: bucket_op_with_timer
2021/12/23 12:36:30 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:36:35 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/23 12:36:35 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/23 12:36:40 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/23 12:36:45 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/23 12:36:50 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/23 12:36:55 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/23 12:36:55 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/23 12:36:56 No eventing-con process running
2021/12/23 12:36:56 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings deleting in background"
}
2021/12/23 12:36:57 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:37:07 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/23 12:37:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:37:09 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:37:19 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/23 12:37:24 Waiting for dst bucket item count to get to: 0 curr count: 12500
2021/12/23 12:37:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopNonDefaultSettings (298.90s)
=== RUN   TestPauseAndResumeWithWorkerCountChange
2021/12/23 12:37:34 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/23 12:37:34 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/23 12:37:34 No eventing-con process running
2021/12/23 12:37:34 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseAndResumeWithWorkerCountChange not found"
}
2021/12/23 12:37:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:37:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:37:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:37:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:37:48 Deploying app: TestPauseAndResumeWithWorkerCountChange. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:37:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange stored in temp store"
}
2021/12/23 12:37:54 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/23 12:37:59 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/23 12:38:04 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/23 12:38:09 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/23 12:38:09 App: TestPauseAndResumeWithWorkerCountChange got deployed
2021/12/23 12:38:09 Starting bucket ops to source bucket
2021/12/23 12:38:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:38:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:38:28 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:38:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:38:38 Waiting for dst bucket item count to get to: 5000 curr count: 926
2021/12/23 12:38:43 Waiting for dst bucket item count to get to: 5000 curr count: 2124
2021/12/23 12:38:48 Waiting for dst bucket item count to get to: 5000 curr count: 3698
2021/12/23 12:38:48 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:38:48 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:38:48 Pausing app: bucket_op_with_timer
2021/12/23 12:38:48 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:38:53 Waiting for function: TestPauseAndResumeWithWorkerCountChange status to change to paused
2021/12/23 12:38:53 Function: TestPauseAndResumeWithWorkerCountChange status changed to paused
2021/12/23 12:38:53 Resuming app: bucket_op_with_timer
2021/12/23 12:38:55 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:38:55 Starting bucket ops to source bucket
2021/12/23 12:39:06 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:39:11 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:39:16 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:39:21 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:39:26 Waiting for dst bucket item count to get to: 10000 curr count: 5790
2021/12/23 12:39:31 Waiting for dst bucket item count to get to: 10000 curr count: 7884
2021/12/23 12:39:36 Waiting for dst bucket item count to get to: 10000 curr count: 9594
2021/12/23 12:39:36 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 12:39:41 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/23 12:39:47 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/23 12:39:52 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/23 12:39:57 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/23 12:39:57 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/23 12:39:57 No eventing-con process running
2021/12/23 12:39:57 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange deleting in background"
}
2021/12/23 12:39:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:40:08 Waiting for dst bucket item count to get to: 0 curr count: 8032
2021/12/23 12:40:13 Waiting for dst bucket item count to get to: 0 curr count: 3032
2021/12/23 12:40:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:40:15 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:40:25 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 12:40:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseAndResumeWithWorkerCountChange (181.09s)
=== RUN   TestPauseResumeWithEventingReb
2021/12/23 12:40:35 Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/23 12:40:35 App: TestPauseResumeWithEventingReb got un-deployed
2021/12/23 12:40:35 No eventing-con process running
2021/12/23 12:40:35 Function: TestPauseResumeWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeWithEventingReb not found"
}
2021/12/23 12:40:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:40:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:40:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:40:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:40:49 Deploying app: TestPauseResumeWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:40:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb stored in temp store"
}
2021/12/23 12:40:55 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/23 12:41:00 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/23 12:41:05 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/23 12:41:10 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/23 12:41:10 App: TestPauseResumeWithEventingReb got deployed
2021/12/23 12:41:10 Starting bucket ops to source bucket
2021/12/23 12:41:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:41:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:41:29 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:41:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:41:39 Waiting for dst bucket item count to get to: 5000 curr count: 1534
2021/12/23 12:41:44 Waiting for dst bucket item count to get to: 5000 curr count: 3067
2021/12/23 12:41:44 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:41:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:41:44 Pausing app: bucket_op_with_timer
2021/12/23 12:41:44 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:41:49 Waiting for function: TestPauseResumeWithEventingReb status to change to paused
2021/12/23 12:41:49 Function: TestPauseResumeWithEventingReb status changed to paused
2021/12/23 12:41:49 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 12:41:50 Removing node(s): [] from the cluster
2021/12/23 12:41:55 Rebalance progress: 29.16666666666666
2021/12/23 12:42:00 Rebalance progress: 29.16666666666666
2021/12/23 12:42:05 Rebalance progress: 100
2021/12/23 12:42:05 Resuming app: bucket_op_with_timer
2021/12/23 12:42:06 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:42:06 Starting bucket ops to source bucket
2021/12/23 12:42:18 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:42:23 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:42:28 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:42:33 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/23 12:42:38 Waiting for dst bucket item count to get to: 10000 curr count: 6301
2021/12/23 12:42:43 Waiting for dst bucket item count to get to: 10000 curr count: 6301
2021/12/23 12:42:43 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 12:42:43 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 12:42:48 Rebalance progress: 29.16666666666666
2021/12/23 12:42:53 Rebalance progress: 29.16666666666666
2021/12/23 12:42:58 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/23 12:43:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:43:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:43:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:43:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:44:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:44:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:44:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:44:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:45:03 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:45:18 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:45:33 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 12:45:48 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:73} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: 4ff8bb7dd12fdd521e15562ebb18285c
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [0-56]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [57-113]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [114-170]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [171-227]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [228-284]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 57
	v: [285-341]
Producer node: 172.31.2.134:9300	Node UUID: e2e1fa79f8570629dc773952c75ea4a3
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [683-739]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [740-796]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [797-853]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [854-910]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [911-967]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [968-1023]
Producer node: 127.0.0.1:9303	Node UUID: bf4fde82ef2cadf6f9903f865d4f953a
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [513-569]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [570-626]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [627-682]
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [342-398]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [399-455]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [456-512]


2021/12/23 12:46:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:46:08 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/23 12:46:13 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/23 12:46:18 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/23 12:46:23 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/23 12:46:23 clusterSetup.go:405: App: TestPauseResumeWithEventingReb got un-deployed
2021/12/23 12:46:23 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:46:23 eventingSetup.go:418: Function: TestPauseResumeWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb deleting in background"
}
2021/12/23 12:46:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:46:35 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 12:46:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:46:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:46:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 12:46:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeWithEventingReb (381.85s)
=== RUN   TestBucketDeleteAfterPause
2021/12/23 12:46:57 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/23 12:46:57 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/23 12:46:57 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:46:57 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteAfterPause not found"
}
2021/12/23 12:46:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:47:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:47:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:47:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:47:11 eventingSetup.go:125: Deploying app: TestBucketDeleteAfterPause. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:47:12 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause stored in temp store"
}
2021/12/23 12:47:17 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/23 12:47:22 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/23 12:47:27 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/23 12:47:32 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/23 12:47:32 clusterSetup.go:354: App: TestBucketDeleteAfterPause got deployed
2021/12/23 12:47:32 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:47:41 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:47:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:47:41 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:47:41 handler_test.go:712: Pausing app: bucket_op_on_update
2021/12/23 12:47:42 eventingSetup.go:270: Function: TestBucketDeleteAfterPause update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:47:47 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to paused
2021/12/23 12:47:47 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to paused
2021/12/23 12:47:47 handler_test.go:716: Deleting source bucket: default
2021/12/23 12:47:53 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to undeployed
2021/12/23 12:47:53 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to undeployed
2021/12/23 12:47:58 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/23 12:47:58 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/23 12:47:58 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:47:58 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause deleting in background"
}
2021/12/23 12:48:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:48:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:48:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:48:16 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:48:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketDeleteAfterPause (89.44s)
=== RUN   TestChangeFnCodeBetweenPauseResume
2021/12/23 12:48:26 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/23 12:48:26 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/23 12:48:26 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:48:26 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestChangeFnCodeBetweenPauseResume not found"
}
2021/12/23 12:48:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:48:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:48:35 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:48:40 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:48:40 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:48:41 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/23 12:48:46 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/23 12:48:51 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/23 12:48:56 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/23 12:49:01 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/23 12:49:01 clusterSetup.go:354: App: TestChangeFnCodeBetweenPauseResume got deployed
2021/12/23 12:49:01 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:49:36 handler_test.go:739: Pausing function: TestChangeFnCodeBetweenPauseResume
2021/12/23 12:49:36 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:49:41 eventingSetup.go:719: Waiting for function: TestChangeFnCodeBetweenPauseResume status to change to paused
2021/12/23 12:49:41 eventingSetup.go:727: Function: TestChangeFnCodeBetweenPauseResume status changed to paused
2021/12/23 12:52:41 handler_test.go:748: Resuming function: TestChangeFnCodeBetweenPauseResume
2021/12/23 12:52:41 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 5000 dst bucket: 0
2021/12/23 12:52:42 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/23 12:52:47 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/23 12:52:47 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:52:47 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 0 expected 5000
2021/12/23 12:52:52 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/23 12:52:52 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:52:52 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 2951 expected 5000
2021/12/23 12:52:57 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/23 12:52:57 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:52:57 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 3901 expected 5000
2021/12/23 12:53:02 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/23 12:53:02 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:53:02 eventingSetup.go:804: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 5000
2021/12/23 12:53:02 handler_test.go:753: Undeploying function: TestChangeFnCodeBetweenPauseResume
2021/12/23 12:53:02 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:53:12 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/23 12:53:17 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/23 12:53:22 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/23 12:53:22 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/23 12:53:22 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:53:22 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume deleting in background"
}
2021/12/23 12:53:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:53:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:53:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 12:53:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:53:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:53:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestChangeFnCodeBetweenPauseResume (328.92s)
=== RUN   TestCommentUnCommentOnDelete
2021/12/23 12:53:55 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/23 12:53:55 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/23 12:53:55 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:53:55 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCommentUnCommentOnDelete not found"
}
2021/12/23 12:53:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:54:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:54:03 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:54:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:54:09 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:54:10 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/23 12:54:15 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:54:20 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:54:25 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:54:30 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:54:30 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/23 12:54:30 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:54:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:54:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:54:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/23 12:54:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:54:49 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:54:49 handler_test.go:780: Undeploying app: TestCommentUnCommentOnDelete
2021/12/23 12:54:49 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:55:19 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 5000 dst bucket: 5000
2021/12/23 12:55:20 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/23 12:55:25 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:55:30 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:55:35 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:55:40 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/23 12:55:40 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/23 12:55:40 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:55:53 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:55:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 270
2021/12/23 12:56:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 270
2021/12/23 12:56:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:56:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:56:03 handler_test.go:799: Undeploying app: TestCommentUnCommentOnDelete
2021/12/23 12:56:03 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 12:56:13 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/23 12:56:18 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/23 12:56:23 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/23 12:56:23 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/23 12:56:23 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:56:23 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete deleting in background"
}
2021/12/23 12:56:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:56:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:56:31 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:56:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCommentUnCommentOnDelete (170.39s)
=== RUN   TestCPPWorkerCleanup
2021/12/23 12:56:46 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/23 12:56:46 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/23 12:56:46 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:56:46 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCPPWorkerCleanup not found"
}
2021/12/23 12:56:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:56:52 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:56:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:56:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:56:59 eventingSetup.go:125: Deploying app: TestCPPWorkerCleanup. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:57:00 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup stored in temp store"
}
2021/12/23 12:57:00 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:57:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:57:12 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:57:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:57:22 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:57:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2461
2021/12/23 12:57:32 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4961
2021/12/23 12:57:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 12:57:32 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 12:57:37 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/23 12:57:42 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/23 12:57:47 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/23 12:57:52 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/23 12:57:52 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/23 12:57:52 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:57:52 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup deleting in background"
}
2021/12/23 12:57:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:58:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 12:58:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 12:58:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:58:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:58:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3792
2021/12/23 12:58:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3792
2021/12/23 12:58:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCPPWorkerCleanup (139.56s)
=== RUN   TestWithUserXattrs
2021/12/23 12:59:05 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/23 12:59:05 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/23 12:59:05 eventingSetup.go:46: No eventing-con process running
2021/12/23 12:59:05 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestWithUserXattrs not found"
}
2021/12/23 12:59:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:59:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:59:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 12:59:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 12:59:18 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:59:20 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:59:22 eventingSetup.go:125: Deploying app: TestWithUserXattrs. Item count src bucket: 0 dst bucket: 0
2021/12/23 12:59:23 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs stored in temp store"
}
2021/12/23 12:59:28 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/23 12:59:33 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/23 12:59:38 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/23 12:59:43 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/23 12:59:43 clusterSetup.go:354: App: TestWithUserXattrs got deployed
2021/12/23 12:59:43 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 12:59:53 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 12:59:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 2178
2021/12/23 13:00:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4678
2021/12/23 13:00:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:00:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:00:08 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/23 13:00:13 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/23 13:00:18 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/23 13:00:23 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/23 13:00:23 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/23 13:00:23 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:00:23 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs deleting in background"
}
2021/12/23 13:00:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:00:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:00:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:00:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:00:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:00:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestWithUserXattrs (110.89s)
=== RUN   TestEventProcessingPostBucketFlush
2021/12/23 13:00:56 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:00:56 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/23 13:00:56 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:00:56 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventProcessingPostBucketFlush not found"
}
2021/12/23 13:00:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:01:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:01:04 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:01:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:01:09 eventingSetup.go:125: Deploying app: TestEventProcessingPostBucketFlush. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:01:10 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush stored in temp store"
}
2021/12/23 13:01:15 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/23 13:01:20 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/23 13:01:25 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/23 13:01:30 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/23 13:01:30 clusterSetup.go:354: App: TestEventProcessingPostBucketFlush got deployed
2021/12/23 13:01:30 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 13:01:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:01:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:01:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:01:45 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:01:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:01:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2520
2021/12/23 13:02:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 20
2021/12/23 13:02:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:02:05 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:02:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:02:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:02:15 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 13:02:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 16049
2021/12/23 13:02:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 16049
2021/12/23 13:02:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/23 13:02:45 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:02:45 eventingSetup.go:270: Function: TestEventProcessingPostBucketFlush update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 13:02:50 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:02:56 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:03:01 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:03:06 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:03:06 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/23 13:03:06 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:03:11 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/23 13:03:11 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/23 13:03:11 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:03:11 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush deleting in background"
}
2021/12/23 13:03:12 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:03:22 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/23 13:03:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/23 13:03:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:03:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:03:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 23810
2021/12/23 13:03:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11310
2021/12/23 13:03:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventProcessingPostBucketFlush (177.58s)
=== RUN   TestMetaBucketDelete
2021/12/23 13:03:54 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/23 13:03:54 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/23 13:03:54 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:03:54 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDelete not found"
}
2021/12/23 13:03:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:04:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:04:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:04:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:04:07 eventingSetup.go:125: Deploying app: TestMetaBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:04:08 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete stored in temp store"
}
2021/12/23 13:04:13 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/23 13:04:18 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/23 13:04:23 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/23 13:04:28 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/23 13:04:28 clusterSetup.go:354: App: TestMetaBucketDelete got deployed
2021/12/23 13:04:28 handler_test.go:902: Deleting metadata bucket: eventing
2021/12/23 13:04:29 handler_test.go:904: Deleted metadata bucket: eventing
2021/12/23 13:04:34 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/23 13:04:34 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/23 13:04:49 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/23 13:04:49 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/23 13:04:49 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:04:49 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete deleting in background"
}
2021/12/23 13:04:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:04:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:04:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:05:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDelete (78.52s)
=== RUN   TestMetaBucketDeleteWithBootstrap
2021/12/23 13:05:12 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:05:12 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:05:12 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:05:12 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDeleteWithBootstrap not found"
}
2021/12/23 13:05:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:05:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:05:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:05:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:05:25 eventingSetup.go:125: Deploying app: TestMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:05:27 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/23 13:05:45 handler_test.go:923: Deleting metadata bucket: eventing
2021/12/23 13:05:46 handler_test.go:925: Deleted metadata bucket: eventing
2021/12/23 13:06:01 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:06:01 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:06:16 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:06:16 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:06:16 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:06:16 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/23 13:06:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:06:22 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:06:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:06:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDeleteWithBootstrap (86.76s)
=== RUN   TestSourceBucketDelete
2021/12/23 13:06:39 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/23 13:06:39 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/23 13:06:39 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:06:39 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDelete not found"
}
2021/12/23 13:06:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:06:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:06:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:06:52 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:06:52 eventingSetup.go:125: Deploying app: TestSourceBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:06:53 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete stored in temp store"
}
2021/12/23 13:06:58 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/23 13:07:03 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/23 13:07:08 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/23 13:07:13 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/23 13:07:13 clusterSetup.go:354: App: TestSourceBucketDelete got deployed
2021/12/23 13:07:13 handler_test.go:945: Deleting source bucket: default
2021/12/23 13:07:15 handler_test.go:947: Deleted source bucket: default
2021/12/23 13:07:20 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/23 13:07:20 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/23 13:07:35 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/23 13:07:35 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/23 13:07:35 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:07:35 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete deleting in background"
}
2021/12/23 13:07:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:07:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:07:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:07:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDelete (78.70s)
=== RUN   TestSourceBucketDeleteWithBootstrap
2021/12/23 13:07:58 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:07:58 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:07:58 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:07:58 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDeleteWithBootstrap not found"
}
2021/12/23 13:07:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:08:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:08:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:08:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:08:11 eventingSetup.go:125: Deploying app: TestSourceBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:08:12 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap stored in temp store"
}
2021/12/23 13:08:31 handler_test.go:966: Deleting source bucket: default
2021/12/23 13:08:32 handler_test.go:968: Deleted source bucket: default
2021/12/23 13:08:47 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:08:47 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:09:02 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:09:02 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:09:02 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:09:03 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap deleting in background"
}
2021/12/23 13:09:04 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:09:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:09:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:09:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDeleteWithBootstrap (87.84s)
=== RUN   TestSourceAndMetaBucketDeleteWithBootstrap
2021/12/23 13:09:26 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:09:26 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:09:26 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:09:26 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceAndMetaBucketDeleteWithBootstrap not found"
}
2021/12/23 13:09:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:09:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:09:34 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:09:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:09:39 eventingSetup.go:125: Deploying app: TestSourceAndMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:09:40 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/23 13:09:59 handler_test.go:988: Deleting source bucket: default
2021/12/23 13:10:00 handler_test.go:990: Deleted source bucket: default
2021/12/23 13:10:00 handler_test.go:992: Deleting metadata bucket: eventing
2021/12/23 13:10:01 handler_test.go:994: Deleted metadata bucket: eventing
2021/12/23 13:10:16 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:10:16 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:10:31 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/23 13:10:31 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/23 13:10:31 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:10:31 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/23 13:10:32 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:10:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:10:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:10:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceAndMetaBucketDeleteWithBootstrap (88.21s)
=== RUN   TestUndeployDuringBootstrap
2021/12/23 13:10:54 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/23 13:10:54 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/23 13:10:54 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:10:54 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployDuringBootstrap not found"
}
2021/12/23 13:10:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:11:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:11:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:11:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:11:07 eventingSetup.go:125: Deploying app: TestUndeployDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:11:08 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap stored in temp store"
}
2021/12/23 13:11:08 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 13:11:20 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:11:20 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/23 13:11:20 clusterSetup.go:378: App: TestUndeployDuringBootstrap started bootstrap
2021/12/23 13:11:20 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/23 13:11:25 clusterSetup.go:367: No apps undergoing bootstrap
2021/12/23 13:11:30 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/23 13:11:35 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/23 13:11:40 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/23 13:11:40 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/23 13:11:40 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:12:05 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/23 13:12:05 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/23 13:12:05 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:12:05 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap deleting in background"
}
2021/12/23 13:12:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:12:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:12:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:12:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:12:28 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2693
2021/12/23 13:12:33 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 279
2021/12/23 13:12:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployDuringBootstrap (109.56s)
=== RUN   TestDeleteBeforeUndeploy
2021/12/23 13:12:38 eventingSetup.go:125: Deploying app: TestDeleteBeforeUndeploy. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:12:40 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy stored in temp store"
}
2021/12/23 13:12:45 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/23 13:12:50 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/23 13:12:55 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/23 13:13:00 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/23 13:13:00 clusterSetup.go:354: App: TestDeleteBeforeUndeploy got deployed
2021/12/23 13:13:00 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 422 dump: {
 "name": "ERR_APP_NOT_UNDEPLOYED",
 "code": 22,
 "description": "Function: TestDeleteBeforeUndeploy skipping delete request as it hasn't been undeployed"
}
2021/12/23 13:13:00 eventingSetup.go:270: Function: TestDeleteBeforeUndeploy update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 13:13:00 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy deleting in background"
}
2021/12/23 13:13:05 clusterSetup.go:397: Waiting for app: TestDeleteBeforeUndeploy to get un-deployed
2021/12/23 13:13:05 clusterSetup.go:405: App: TestDeleteBeforeUndeploy got un-deployed
2021/12/23 13:13:05 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:13:05 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeleteBeforeUndeploy not found"
}
2021/12/23 13:13:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:13:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:13:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:13:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDeleteBeforeUndeploy (44.53s)
=== RUN   TestUndeployWhenTimersAreFired
2021/12/23 13:13:28 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:13:28 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/23 13:13:28 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:13:28 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWhenTimersAreFired not found"
}
2021/12/23 13:13:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:13:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:13:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:13:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:13:41 eventingSetup.go:125: Deploying app: TestUndeployWhenTimersAreFired. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:13:43 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired stored in temp store"
}
2021/12/23 13:13:48 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/23 13:13:53 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/23 13:13:58 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/23 13:14:03 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/23 13:14:03 clusterSetup.go:354: App: TestUndeployWhenTimersAreFired got deployed
2021/12/23 13:14:03 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 13:14:38 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:14:43 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:14:48 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:14:53 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:14:53 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/23 13:14:53 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:16:38 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/23 13:16:38 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/23 13:16:38 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:16:38 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired deleting in background"
}
2021/12/23 13:16:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:16:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:16:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 13:16:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:16:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:17:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:17:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWhenTimersAreFired (228.16s)
=== RUN   TestUndeployWithKVFailover
2021/12/23 13:17:16 clusterSetup.go:397: Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/23 13:17:16 clusterSetup.go:405: App: TestUndeployWithKVFailover got un-deployed
2021/12/23 13:17:16 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:17:16 eventingSetup.go:418: Function: TestUndeployWithKVFailover delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWithKVFailover not found"
}
2021/12/23 13:17:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:17:22 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:17:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:17:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:17:29 eventingSetup.go:125: Deploying app: TestUndeployWithKVFailover. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:17:30 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover stored in temp store"
}
2021/12/23 13:17:30 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 13:17:37 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:17:42 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:17:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:17:52 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:17:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:18:02 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:18:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4051
2021/12/23 13:18:12 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4051
2021/12/23 13:18:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:18:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/23 13:18:13 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 13:18:18 Rebalance progress: 29.16666666666666
2021/12/23 13:18:23 Rebalance progress: 29.16666666666666
2021/12/23 13:18:28 Rebalance progress: 100
2021/12/23 13:18:28 Failing over: [127.0.0.1:19003]
2021/12/23 13:18:38 Function: TestUndeployWithKVFailover update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/23 13:19:38 Removing node(s): [] from the cluster
2021/12/23 13:19:43 Rebalance progress: 29.16666666666666
2021/12/23 13:19:48 Rebalance progress: 29.16666666666666
2021/12/23 13:19:53 Rebalance progress: 100
2021/12/23 13:19:58 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/23 13:19:58 App: TestUndeployWithKVFailover got un-deployed
2021/12/23 13:19:58 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:20:03 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/23 13:20:03 App: TestUndeployWithKVFailover got un-deployed
2021/12/23 13:20:03 No eventing-con process running
2021/12/23 13:20:03 Function: TestUndeployWithKVFailover delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover deleting in background"
}
2021/12/23 13:20:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:20:15 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:20:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:20:16 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:20:26 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:20:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWithKVFailover (200.32s)
=== RUN   TestBucketFlushWhileFnDeployed
2021/12/23 13:20:36 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/23 13:20:36 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/23 13:20:36 No eventing-con process running
2021/12/23 13:20:36 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketFlushWhileFnDeployed not found"
}
2021/12/23 13:20:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:20:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:20:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:20:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:20:50 Deploying app: TestBucketFlushWhileFnDeployed. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:20:51 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed stored in temp store"
}
2021/12/23 13:20:56 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/23 13:21:01 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/23 13:21:06 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/23 13:21:11 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/23 13:21:11 App: TestBucketFlushWhileFnDeployed got deployed
2021/12/23 13:21:11 Starting bucket ops to source bucket
2021/12/23 13:21:31 Waiting for dst bucket item count to get to: 20000 curr count: 9676
2021/12/23 13:21:31 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/23 13:21:31 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:21:34 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:21:44 Waiting for dst bucket item count to get to: 0 curr count: 20000
2021/12/23 13:21:49 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 13:21:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:21:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:22:00 Waiting for dst bucket item count to get to: 0 curr count: 15318
2021/12/23 13:22:05 Waiting for dst bucket item count to get to: 0 curr count: 15318
2021/12/23 13:22:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:22:05 Starting bucket ops to source bucket
2021/12/23 13:22:18 Waiting for dst bucket item count to get to: 10000 curr count: 3757
2021/12/23 13:22:23 Waiting for dst bucket item count to get to: 10000 curr count: 3757
2021/12/23 13:22:23 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 13:22:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:22:28 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/23 13:22:33 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/23 13:22:38 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/23 13:22:43 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/23 13:22:43 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/23 13:22:43 No eventing-con process running
2021/12/23 13:22:43 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed deleting in background"
}
2021/12/23 13:22:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:22:55 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 13:22:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:22:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:23:06 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 13:23:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketFlushWhileFnDeployed (160.00s)
=== RUN   TestTimerOverwrite
2021/12/23 13:23:16 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/23 13:23:16 App: TestTimerOverwrite got un-deployed
2021/12/23 13:23:16 No eventing-con process running
2021/12/23 13:23:16 Function: TestTimerOverwrite delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverwrite not found"
}
2021/12/23 13:23:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:23:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:23:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:23:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:23:29 Deploying app: TestTimerOverwrite. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:23:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite stored in temp store"
}
2021/12/23 13:23:36 Waiting for app: TestTimerOverwrite to get deployed
2021/12/23 13:23:41 Waiting for app: TestTimerOverwrite to get deployed
2021/12/23 13:23:46 Waiting for app: TestTimerOverwrite to get deployed
2021/12/23 13:23:51 Waiting for app: TestTimerOverwrite to get deployed
2021/12/23 13:23:51 App: TestTimerOverwrite got deployed
2021/12/23 13:24:01 Starting bucket ops to source bucket
2021/12/23 13:24:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:50 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:24:55 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:25:00 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:25:05 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:25:05 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:25:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:25:20 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/23 13:25:25 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/23 13:25:30 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/23 13:25:35 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/23 13:25:35 App: TestTimerOverwrite got un-deployed
2021/12/23 13:25:35 No eventing-con process running
2021/12/23 13:25:35 Function: TestTimerOverwrite delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite deleting in background"
}
2021/12/23 13:25:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:25:47 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:25:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:25:49 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:25:59 Waiting for dst bucket item count to get to: 0 curr count: 4379
2021/12/23 13:26:04 Waiting for dst bucket item count to get to: 0 curr count: 1892
2021/12/23 13:26:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverwrite (177.18s)
=== RUN   TestUndeployBackdoorDuringBootstrap
2021/12/23 13:26:09 Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/23 13:26:09 Removing node(s): [] from the cluster
2021/12/23 13:26:14 Rebalance progress: 100
2021/12/23 13:26:14 Failing over: [127.0.0.1:19003]
2021/12/23 13:26:19 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/23 13:26:19 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/23 13:26:19 No eventing-con process running
2021/12/23 13:26:19 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/23 13:26:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:26:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:26:27 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:26:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:26:32 Deploying app: TestUndeployBackdoorDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:26:33 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap stored in temp store"
}
2021/12/23 13:26:33 Starting bucket ops to source bucket
2021/12/23 13:26:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:26:43 Context: Post to set retry counter, response code: 200 dump: {
 "code": 0
}
2021/12/23 13:26:43 set retry counter, response &{[123 10 32 34 99 111 100 101 34 58 32 48 10 125] }
2021/12/23 13:27:48 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/23 13:27:48 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/23 13:27:48 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:27:48 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap deleting in background"
}
2021/12/23 13:27:53 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/23 13:27:53 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/23 13:27:53 No eventing-con process running
2021/12/23 13:27:53 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/23 13:27:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:28:05 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:28:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:28:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:28:16 Waiting for dst bucket item count to get to: 0 curr count: 2013
2021/12/23 13:28:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:28:16 Removing node(s): [] from the cluster
2021/12/23 13:28:21 Rebalance progress: 100
--- PASS: TestUndeployBackdoorDuringBootstrap (137.82s)
=== RUN   TestOnUpdateSrcMutation
2021/12/23 13:28:31 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/23 13:28:31 App: TestOnUpdateSrcMutation got un-deployed
2021/12/23 13:28:31 No eventing-con process running
2021/12/23 13:28:31 Function: TestOnUpdateSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutation not found"
}
2021/12/23 13:28:33 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:28:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:28:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:28:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:28:44 Deploying app: TestOnUpdateSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:28:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation stored in temp store"
}
2021/12/23 13:28:46 Starting bucket ops to source bucket
2021/12/23 13:29:13 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 13:29:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:29:18 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/23 13:29:23 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/23 13:29:28 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/23 13:29:33 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/23 13:29:33 App: TestOnUpdateSrcMutation got un-deployed
2021/12/23 13:29:33 No eventing-con process running
2021/12/23 13:29:33 Function: TestOnUpdateSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation deleting in background"
}
2021/12/23 13:29:34 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:29:44 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 13:29:49 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:29:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:29:51 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:29:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutation (94.49s)
=== RUN   TestOnDeleteSrcMutation
2021/12/23 13:30:06 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/23 13:30:06 App: TestOnDeleteSrcMutation got un-deployed
2021/12/23 13:30:06 No eventing-con process running
2021/12/23 13:30:06 Function: TestOnDeleteSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutation not found"
}
2021/12/23 13:30:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:30:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:30:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:30:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:30:19 Deploying app: TestOnDeleteSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:30:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation stored in temp store"
}
2021/12/23 13:30:20 Starting bucket ops to source bucket
2021/12/23 13:30:49 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:30:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:30:54 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/23 13:30:59 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/23 13:31:04 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/23 13:31:09 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/23 13:31:09 App: TestOnDeleteSrcMutation got un-deployed
2021/12/23 13:31:09 No eventing-con process running
2021/12/23 13:31:09 Function: TestOnDeleteSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation deleting in background"
}
2021/12/23 13:31:10 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:31:20 Waiting for dst bucket item count to get to: 0 curr count: 4243
2021/12/23 13:31:25 Waiting for dst bucket item count to get to: 0 curr count: 4243
2021/12/23 13:31:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:31:27 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:31:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutation (96.21s)
=== RUN   TestOnUpdateSrcMutationWithTimer
2021/12/23 13:31:42 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/23 13:31:42 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/23 13:31:42 No eventing-con process running
2021/12/23 13:31:42 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutationWithTimer not found"
}
2021/12/23 13:31:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:31:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:31:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:31:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:31:55 Deploying app: TestOnUpdateSrcMutationWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:31:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer stored in temp store"
}
2021/12/23 13:31:56 Starting bucket ops to source bucket
2021/12/23 13:32:33 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 13:32:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:32:39 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/23 13:32:44 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/23 13:32:49 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/23 13:32:54 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/23 13:32:54 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/23 13:32:54 No eventing-con process running
2021/12/23 13:32:54 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer deleting in background"
}
2021/12/23 13:32:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:33:05 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 13:33:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:33:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:33:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutationWithTimer (99.65s)
=== RUN   TestOnDeleteSrcMutationsWithTimer
2021/12/23 13:33:22 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/23 13:33:22 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/23 13:33:22 No eventing-con process running
2021/12/23 13:33:22 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutationsWithTimer not found"
}
2021/12/23 13:33:23 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:33:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:33:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:33:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:33:35 Deploying app: TestOnDeleteSrcMutationsWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:33:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer stored in temp store"
}
2021/12/23 13:33:36 Starting bucket ops to source bucket
2021/12/23 13:34:25 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:34:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:34:30 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/23 13:34:35 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/23 13:34:40 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/23 13:34:45 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/23 13:34:45 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/23 13:34:45 No eventing-con process running
2021/12/23 13:34:45 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer deleting in background"
}
2021/12/23 13:34:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:34:56 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 13:34:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:34:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:35:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutationsWithTimer (111.34s)
=== RUN   TestInterHandlerRecursion
2021/12/23 13:35:13 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:35:13 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/23 13:35:13 No eventing-con process running
2021/12/23 13:35:13 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/23 13:35:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:35:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:35:21 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:35:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:35:31 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:35:31 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/23 13:35:31 No eventing-con process running
2021/12/23 13:35:31 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/23 13:35:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:35:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:35:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:35:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:35:44 Deploying app: TestInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:35:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/23 13:35:45 response body {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/23 13:35:50 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:35:55 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:36:00 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:36:05 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:36:05 App: TestInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/23 13:36:05 Deploying app: TestInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:36:05 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
}
2021/12/23 13:36:05 response body {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
} err 
2021/12/23 13:36:11 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:36:16 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:36:21 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:36:26 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:36:26 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/23 13:36:26 No eventing-con process running
2021/12/23 13:36:26 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/23 13:36:27 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:36:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:36:34 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:36:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:36:44 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:36:44 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/23 13:36:44 No eventing-con process running
2021/12/23 13:36:44 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/23 13:36:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:36:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:36:52 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:36:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterHandlerRecursion (113.87s)
=== RUN   TestInterBucketRecursion
2021/12/23 13:37:02 Deploying app: TestInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:37:03 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
}
2021/12/23 13:37:03 response body {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/23 13:37:03 Deploying app: TestInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:37:03 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion_function2 causes a cycle involving functions: [TestInterBucketRecursion_function1], hence deployment is disallowed"
}
2021/12/23 13:37:08 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/23 13:37:13 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/23 13:37:18 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/23 13:37:23 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/23 13:37:23 App: TestInterBucketRecursion_function1 got deployed
2021/12/23 13:37:28 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:37:33 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:37:38 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:37:43 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:37:43 App: TestInterBucketRecursion_function1 got un-deployed
2021/12/23 13:37:43 No eventing-con process running
2021/12/23 13:37:43 Function: TestInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 deleting in background"
}
2021/12/23 13:37:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:37:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:37:52 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:37:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:38:02 Waiting for app: TestInterBucketRecursion_function2 to get un-deployed
2021/12/23 13:38:02 App: TestInterBucketRecursion_function2 got un-deployed
2021/12/23 13:38:02 No eventing-con process running
2021/12/23 13:38:02 Function: TestInterBucketRecursion_function2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion_function2 not found"
}
2021/12/23 13:38:03 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:38:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:38:10 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:38:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion (77.71s)
=== RUN   TestLargeHandler
2021/12/23 13:38:15 Changed force_compress value to false
2021/12/23 13:38:15 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:38:15 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_APPCODE_SIZE",
 "code": 39,
 "description": "Function: TestLargeHandler_function handler Code size is more than 131072. Code Size: 131768"
}
2021/12/23 13:38:15 Changed force_compress value to true
2021/12/23 13:38:15 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:38:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function stored in temp store"
}
2021/12/23 13:38:22 Waiting for app: TestLargeHandler_function to get deployed
2021/12/23 13:38:27 Waiting for app: TestLargeHandler_function to get deployed
2021/12/23 13:38:32 Waiting for app: TestLargeHandler_function to get deployed
2021/12/23 13:38:37 Waiting for app: TestLargeHandler_function to get deployed
2021/12/23 13:38:37 App: TestLargeHandler_function got deployed
2021/12/23 13:38:42 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/23 13:38:47 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/23 13:38:52 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/23 13:38:57 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/23 13:38:57 App: TestLargeHandler_function got un-deployed
2021/12/23 13:38:57 No eventing-con process running
2021/12/23 13:38:57 Function: TestLargeHandler_function delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function deleting in background"
}
2021/12/23 13:38:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:39:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestLargeHandler (55.48s)
=== RUN   TestAllowInterHandlerRecursion
2021/12/23 13:39:20 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:39:20 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/23 13:39:20 No eventing-con process running
2021/12/23 13:39:20 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/23 13:39:21 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:39:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:39:38 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:39:38 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/23 13:39:38 No eventing-con process running
2021/12/23 13:39:38 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/23 13:39:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:39:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:39:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:39:51 Deploying app: TestAllowInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:39:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/23 13:39:52 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/23 13:39:57 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:40:02 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:40:07 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:40:12 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/23 13:40:12 App: TestAllowInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/23 13:40:12 Deploying app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:40:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
}
2021/12/23 13:40:14 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
} err 
2021/12/23 13:40:19 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/23 13:40:24 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/23 13:40:29 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/23 13:40:34 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/23 13:40:34 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got deployed
2021/12/23 13:40:34 TestAllowInterHandlerRecursion_src_bucket_op_on_delete is deployed
2021/12/23 13:40:39 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:40:44 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:40:49 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:40:54 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/23 13:40:54 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/23 13:40:54 No eventing-con process running
2021/12/23 13:40:54 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/23 13:40:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:41:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:41:03 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:41:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:41:13 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:41:18 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:41:23 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:41:28 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/23 13:41:28 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/23 13:41:28 No eventing-con process running
2021/12/23 13:41:28 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete deleting in background"
}
2021/12/23 13:41:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:41:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:41:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:41:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterHandlerRecursion (151.23s)
=== RUN   TestAllowInterBucketRecursion
2021/12/23 13:41:46 Deploying app: TestAllowInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:41:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
}
2021/12/23 13:41:48 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/23 13:41:53 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/23 13:41:58 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/23 13:42:03 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/23 13:42:08 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/23 13:42:08 App: TestAllowInterBucketRecursion_function1 got deployed
2021/12/23 13:42:08 Deploying app: TestAllowInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:42:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
}
2021/12/23 13:42:09 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
} err 
2021/12/23 13:42:14 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/23 13:42:19 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/23 13:42:24 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/23 13:42:29 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/23 13:42:29 App: TestAllowInterBucketRecursion_function2 got deployed
2021/12/23 13:42:29 TestAllowInterBucketRecursion_function2 is deployed
2021/12/23 13:42:34 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:42:39 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:42:44 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:42:49 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/23 13:42:49 App: TestAllowInterBucketRecursion_function1 got un-deployed
2021/12/23 13:42:49 No eventing-con process running
2021/12/23 13:42:49 Function: TestAllowInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 deleting in background"
}
2021/12/23 13:42:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:42:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:42:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:43:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:43:08 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/23 13:43:13 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/23 13:43:18 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/23 13:43:23 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/23 13:43:23 App: TestAllowInterBucketRecursion_function2 got un-deployed
2021/12/23 13:43:23 No eventing-con process running
2021/12/23 13:43:23 Function: TestAllowInterBucketRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 deleting in background"
}
2021/12/23 13:43:25 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:43:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:43:32 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:43:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterBucketRecursion (115.29s)
=== RUN   TestInterBucketRecursion2
2021/12/23 13:43:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:43:49 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:43:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:44:00 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:44:05 Deploying app: TestInterBucketRecursion2_function1. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:44:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 stored in temp store"
}
2021/12/23 13:44:12 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/23 13:44:17 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/23 13:44:22 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/23 13:44:27 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/23 13:44:27 App: TestInterBucketRecursion2_function1 got deployed
2021/12/23 13:44:27 Deploying app: TestInterBucketRecursion2_function2. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:44:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 stored in temp store"
}
2021/12/23 13:44:33 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/23 13:44:38 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/23 13:44:43 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/23 13:44:48 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/23 13:44:48 App: TestInterBucketRecursion2_function2 got deployed
2021/12/23 13:44:48 Deploying app: TestInterBucketRecursion2_function3. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:44:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 stored in temp store"
}
2021/12/23 13:44:54 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/23 13:44:59 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/23 13:45:04 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/23 13:45:09 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/23 13:45:09 App: TestInterBucketRecursion2_function3 got deployed
2021/12/23 13:45:09 Deploying app: TestInterBucketRecursion2_function4. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:45:09 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion2_function4 causes a cycle involving functions: [TestInterBucketRecursion2_function1 TestInterBucketRecursion2_function2 TestInterBucketRecursion2_function3], hence deployment is disallowed"
}
2021/12/23 13:45:14 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/23 13:45:19 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/23 13:45:24 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/23 13:45:29 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/23 13:45:29 App: TestInterBucketRecursion2_function1 got un-deployed
2021/12/23 13:45:29 No eventing-con process running
2021/12/23 13:45:29 Function: TestInterBucketRecursion2_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 deleting in background"
}
2021/12/23 13:45:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:45:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:45:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:45:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:45:48 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/23 13:45:53 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/23 13:45:58 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/23 13:46:03 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/23 13:46:03 App: TestInterBucketRecursion2_function2 got un-deployed
2021/12/23 13:46:03 No eventing-con process running
2021/12/23 13:46:03 Function: TestInterBucketRecursion2_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 deleting in background"
}
2021/12/23 13:46:04 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:46:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:46:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:46:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:46:21 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/23 13:46:26 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/23 13:46:31 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/23 13:46:36 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/23 13:46:36 App: TestInterBucketRecursion2_function3 got un-deployed
2021/12/23 13:46:36 No eventing-con process running
2021/12/23 13:46:36 Function: TestInterBucketRecursion2_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 deleting in background"
}
2021/12/23 13:46:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:46:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:46:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:46:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:46:55 Waiting for app: TestInterBucketRecursion2_function4 to get un-deployed
2021/12/23 13:46:55 App: TestInterBucketRecursion2_function4 got un-deployed
2021/12/23 13:46:55 No eventing-con process running
2021/12/23 13:46:55 Function: TestInterBucketRecursion2_function4 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion2_function4 not found"
}
2021/12/23 13:46:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:47:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:47:03 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:47:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion2 (213.03s)
=== RUN   TestBucketDeleteWithRebOut
2021/12/23 13:47:15 Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/23 13:47:20 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/23 13:47:23 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:47:38 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:47:53 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:48:08 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:48:23 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:48:38 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:48:53 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:49:08 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:49:23 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:49:38 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/23 13:49:53 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:51} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:

vbucket curr owner:


2021/12/23 13:50:08 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:50:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 13:50:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 13:50:13 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/23 13:50:13 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:50:13 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 13:50:14 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 13:50:19 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/23 13:50:19 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:50:24 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/23 13:50:24 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/23 13:50:24 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:50:24 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteWithRebOut not found"
}
2021/12/23 13:50:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:50:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:50:32 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:50:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:50:42 eventingSetup.go:125: Deploying app: TestBucketDeleteWithRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:50:43 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut stored in temp store"
}
2021/12/23 13:50:48 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/23 13:50:53 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/23 13:50:58 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/23 13:51:03 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/23 13:51:03 clusterSetup.go:354: App: TestBucketDeleteWithRebOut got deployed

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: 4ff8bb7dd12fdd521e15562ebb18285c
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]
Producer node: 172.31.2.134:9300	Node UUID: e2e1fa79f8570629dc773952c75ea4a3
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
Producer node: 127.0.0.1:9303	Node UUID: bf4fde82ef2cadf6f9903f865d4f953a
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]


2021/12/23 13:51:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:51:03 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 13:51:08 Rebalance progress: 29.16666666666666
2021/12/23 13:51:13 Rebalance progress: 29.16666666666666
2021/12/23 13:51:18 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 172.31.2.134:9300	Node UUID: e2e1fa79f8570629dc773952c75ea4a3
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]
Producer node: 127.0.0.1:9303	Node UUID: bf4fde82ef2cadf6f9903f865d4f953a
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]
Producer node: 127.0.0.1:9302	Node UUID: 4ff8bb7dd12fdd521e15562ebb18285c
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]


2021/12/23 13:51:18 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:51:18 handler_test.go:1674: Deleting source bucket: default
2021/12/23 13:51:20 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 13:51:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 13:51:25 Rebalance progress: 29.16666666666666
2021/12/23 13:51:30 Rebalance progress: 29.16666666666666
2021/12/23 13:51:35 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 900
	vb list dump: [0-4, 6, 8-10, 12-31, 33, 35-36, 38-60, 62-67, 69, 71, 73-85, 87-97, 99-111, 114-117, 119-122, 125-130, 132-137, 139-147, 149-163, 165-174, 176-177, 179-183, 185-194, 196-200, 202-214, 216-239, 241-254, 256-265, 267-273, 275-278, 281, 283, 285-295, 297-299, 301-305, 307-318, 320-326, 328-339, 341-351, 353, 355-360, 362-366, 368-377, 380-388, 390-394, 396, 398-405, 407, 409-415, 417-436, 438-452, 454-465, 468, 473, 475-477, 479-484, 488, 490-492, 494-497, 499, 501-506, 508-577, 579-581, 583-592, 594-610, 612-614, 616, 618-625, 628-633, 635-656, 658-664, 666, 668-677, 679-688, 690-695, 697-707, 709-715, 717-724, 726-741, 743-748, 750, 753-763, 766-768, 770, 772-796, 798-818, 821-830, 833-844, 847, 849-851, 853-856, 859-874, 876-894, 896-912, 914-919, 921-922, 924-934, 936-944, 946, 949-963, 965, 968-983, 985-986, 988-993, 996-1023]
	stream status: stopped
	len: 124
	vb list dump: [5, 7, 11, 32, 34, 37, 61, 68, 70, 72, 86, 98, 112-113, 118, 123-124, 131, 138, 148, 164, 175, 178, 184, 195, 201, 215, 240, 255, 266, 274, 279-280, 282, 284, 296, 300, 306, 319, 327, 340, 352, 354, 361, 367, 378-379, 389, 395, 397, 406, 408, 416, 437, 453, 466-467, 469-472, 474, 478, 485-487, 489, 493, 498, 500, 507, 578, 582, 593, 611, 615, 617, 626-627, 634, 657, 665, 667, 678, 689, 696, 708, 716, 725, 742, 749, 751-752, 764-765, 769, 771, 797, 819-820, 831-832, 845-846, 848, 852, 857-858, 875, 895, 913, 920, 923, 935, 945, 947-948, 964, 966-967, 984, 987, 994-995]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: 4ff8bb7dd12fdd521e15562ebb18285c
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 100
	v: [0-4, 6, 8-10, 12-31, 33, 35-36, 38-60, 62-67, 69, 71, 73-85, 87-97, 99-111]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 101
	v: [114-117, 119-122, 125-130, 132-137, 139-147, 149-163, 165-174, 176-177, 179-183, 185-194, 196-200, 202-214, 216-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 100
	v: [228-239, 241-254, 256-265, 267-273, 275-278, 281, 283, 285-295, 297-299, 301-305, 307-318, 320-326, 328-339, 341]
Producer node: 172.31.2.134:9300	Node UUID: e2e1fa79f8570629dc773952c75ea4a3
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 99
	v: [911-912, 914-919, 921-922, 924-934, 936-944, 946, 949-963, 965, 968-983, 985-986, 988-993, 996-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 101
	v: [683-688, 690-695, 697-707, 709-715, 717-724, 726-741, 743-748, 750, 753-763, 766-768, 770, 772-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 101
	v: [798-818, 821-830, 833-844, 847, 849-851, 853-856, 859-874, 876-894, 896-910]
Producer node: 127.0.0.1:9303	Node UUID: bf4fde82ef2cadf6f9903f865d4f953a
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 100
	v: [342-351, 353, 355-360, 362-366, 368-377, 380-388, 390-394, 396, 398-405, 407, 409-415, 417-436, 438-452, 454-455]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 98
	v: [456-465, 468, 473, 475-477, 479-484, 488, 490-492, 494-497, 499, 501-506, 508-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 100
	v: [570-577, 579-581, 583-592, 594-610, 612-614, 616, 618-625, 628-633, 635-656, 658-664, 666, 668-677, 679-682]


2021/12/23 13:51:35 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 13:51:40 eventingSetup.go:719: Waiting for function: TestBucketDeleteWithRebOut status to change to undeployed
2021/12/23 13:51:40 eventingSetup.go:727: Function: TestBucketDeleteWithRebOut status changed to undeployed
2021/12/23 13:51:45 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/23 13:51:45 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/23 13:51:45 eventingSetup.go:46: No eventing-con process running
2021/12/23 13:51:45 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut deleting in background"
}
2021/12/23 13:51:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:51:52 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:51:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:51:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:51:58 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/23 13:51:59 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 13:52:04 Rebalance progress: 100
--- PASS: TestBucketDeleteWithRebOut (296.94s)
=== RUN   TestN1QLRecursion
2021/12/23 13:52:07 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 13:52:07 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/23 13:52:07 Removing node(s): [] from the cluster
2021/12/23 13:52:12 Rebalance progress: 100
2021/12/23 13:52:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:52:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:52:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:52:35 Deploying app: TestN1QLRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:52:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
}
2021/12/23 13:52:36 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
} err 
2021/12/23 13:52:41 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/23 13:52:46 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/23 13:52:51 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/23 13:52:56 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/23 13:52:56 App: TestN1QLRecursion_function1 got deployed
2021/12/23 13:52:56 TestN1QLRecursion_function1 is deployed
2021/12/23 13:52:56 Deploying app: TestN1QLRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:52:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
}
2021/12/23 13:52:57 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
} err 
2021/12/23 13:53:02 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/23 13:53:07 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/23 13:53:12 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/23 13:53:17 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/23 13:53:17 App: TestN1QLRecursion_function3 got deployed
2021/12/23 13:53:17 TestN1QLRecursion_function3 is deployed
2021/12/23 13:53:17 Deploying app: TestN1QLRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:53:17 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
}
2021/12/23 13:53:17 response body {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
} err 
2021/12/23 13:53:17 Success: TestN1QLRecursion_function2 is not deployed
2021/12/23 13:53:22 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/23 13:53:28 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/23 13:53:33 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/23 13:53:38 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/23 13:53:38 App: TestN1QLRecursion_function1 got un-deployed
2021/12/23 13:53:38 No eventing-con process running
2021/12/23 13:53:38 Function: TestN1QLRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 deleting in background"
}
2021/12/23 13:53:39 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:53:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:53:46 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:53:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:53:56 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/23 13:54:02 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/23 13:54:07 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/23 13:54:12 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/23 13:54:12 App: TestN1QLRecursion_function3 got un-deployed
2021/12/23 13:54:12 No eventing-con process running
2021/12/23 13:54:12 Function: TestN1QLRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 deleting in background"
}
2021/12/23 13:54:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:54:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:54:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:54:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:54:27 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 13:54:32 Rebalance progress: 100
--- PASS: TestN1QLRecursion (145.11s)
=== RUN   TestN1QLAllowRecursion
2021/12/23 13:54:32 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 13:54:32 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/23 13:54:32 Removing node(s): [] from the cluster
2021/12/23 13:54:37 Rebalance progress: 100
2021/12/23 13:54:43 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:54:49 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:54:55 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:55:00 Deploying app: TestN1QLAllowRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:55:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
}
2021/12/23 13:55:02 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
} err 
2021/12/23 13:55:07 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/23 13:55:12 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/23 13:55:17 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/23 13:55:22 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/23 13:55:22 App: TestN1QLAllowRecursion_function1 got deployed
2021/12/23 13:55:22 TestN1QLAllowRecursion_function1 is deployed
2021/12/23 13:55:22 Deploying app: TestN1QLAllowRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:55:23 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
}
2021/12/23 13:55:23 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
} err 
2021/12/23 13:55:28 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/23 13:55:33 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/23 13:55:38 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/23 13:55:43 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/23 13:55:43 App: TestN1QLAllowRecursion_function3 got deployed
2021/12/23 13:55:43 TestN1QLAllowRecursion_function3 is deployed
2021/12/23 13:55:43 Deploying app: TestN1QLAllowRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:55:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
}
2021/12/23 13:55:44 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
} err 
2021/12/23 13:55:49 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/23 13:55:54 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/23 13:55:59 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/23 13:56:04 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/23 13:56:04 App: TestN1QLAllowRecursion_function2 got deployed
2021/12/23 13:56:04 Success: TestN1QLAllowRecursion_function2 is deployed after allow_interbucket_recursion
2021/12/23 13:56:09 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/23 13:56:14 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/23 13:56:19 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/23 13:56:24 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/23 13:56:24 App: TestN1QLAllowRecursion_function1 got un-deployed
2021/12/23 13:56:25 No eventing-con process running
2021/12/23 13:56:25 Function: TestN1QLAllowRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 deleting in background"
}
2021/12/23 13:56:26 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:56:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:56:33 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:56:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:56:43 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/23 13:56:48 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/23 13:56:53 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/23 13:56:58 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/23 13:56:58 App: TestN1QLAllowRecursion_function2 got un-deployed
2021/12/23 13:56:59 No eventing-con process running
2021/12/23 13:56:59 Function: TestN1QLAllowRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 deleting in background"
}
2021/12/23 13:57:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:57:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:57:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:57:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:57:18 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/23 13:57:23 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/23 13:57:28 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/23 13:57:33 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/23 13:57:33 App: TestN1QLAllowRecursion_function3 got un-deployed
2021/12/23 13:57:33 No eventing-con process running
2021/12/23 13:57:33 Function: TestN1QLAllowRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 deleting in background"
}
2021/12/23 13:57:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:57:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:57:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:57:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:57:48 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 13:57:53 Rebalance progress: 100
--- PASS: TestN1QLAllowRecursion (201.02s)
=== RUN   TestOnDeleteExpiryBucketOp
2021/12/23 13:57:53 Sleeping for some time
2021/12/23 13:58:03 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/23 13:58:03 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/23 13:58:03 No eventing-con process running
2021/12/23 13:58:03 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteExpiryBucketOp not found"
}
2021/12/23 13:58:05 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:58:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:58:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 13:58:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 13:58:16 Deploying app: TestOnDeleteExpiryBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 13:58:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp stored in temp store"
}
2021/12/23 13:58:17 Starting bucket ops to source bucket
2021/12/23 13:59:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:59:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:59:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 13:59:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 13:59:17 Starting bucket ops to source bucket
2021/12/23 13:59:18 Starting bucket ops to source bucket
2021/12/23 14:00:23 Waiting for dst bucket item count to get to: 7000 curr count: 5000
2021/12/23 14:00:28 Waiting for dst bucket item count to get to: 7000 curr count: 5000
2021/12/23 14:00:33 Waiting for dst bucket item count to get to: 7000 curr count: 6000
2021/12/23 14:00:33 src & dst bucket item count matched up. src bucket count: 7000 dst bucket count: 7000
2021/12/23 14:00:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:00:39 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/23 14:00:44 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/23 14:00:49 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/23 14:00:54 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/23 14:00:54 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/23 14:00:54 No eventing-con process running
2021/12/23 14:00:54 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp deleting in background"
}
2021/12/23 14:00:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:01:06 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:01:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:01:07 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:01:17 Waiting for dst bucket item count to get to: 0 curr count: 3500
2021/12/23 14:01:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteExpiryBucketOp (204.73s)
=== RUN   TestTimerOverWriteSameReference
2021/12/23 14:01:22 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/23 14:01:22 App: TestTimerOverWriteSameReference got un-deployed
2021/12/23 14:01:22 No eventing-con process running
2021/12/23 14:01:22 Function: TestTimerOverWriteSameReference delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverWriteSameReference not found"
}
2021/12/23 14:01:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:01:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:01:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:01:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:01:36 Deploying app: TestTimerOverWriteSameReference. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:01:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference stored in temp store"
}
2021/12/23 14:01:42 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/23 14:01:47 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/23 14:01:52 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/23 14:01:57 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/23 14:01:57 App: TestTimerOverWriteSameReference got deployed
2021/12/23 14:02:07 Starting bucket ops to source bucket
2021/12/23 14:03:08 src & dst bucket item count matched up. src bucket count: 1 dst bucket count: 1
2021/12/23 14:03:18 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:03:24 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/23 14:03:29 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/23 14:03:34 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/23 14:03:39 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/23 14:03:39 App: TestTimerOverWriteSameReference got un-deployed
2021/12/23 14:03:39 No eventing-con process running
2021/12/23 14:03:39 Function: TestTimerOverWriteSameReference delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference deleting in background"
}
2021/12/23 14:03:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:03:51 Waiting for dst bucket item count to get to: 0 curr count: 844
2021/12/23 14:03:56 Waiting for dst bucket item count to get to: 0 curr count: 844
2021/12/23 14:03:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:03:57 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:04:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverWriteSameReference (164.92s)
=== RUN   TestCancelTimerBucketop
2021/12/23 14:04:07 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/23 14:04:07 App: TestCancelTimerBucketop got un-deployed
2021/12/23 14:04:07 No eventing-con process running
2021/12/23 14:04:07 Function: TestCancelTimerBucketop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCancelTimerBucketop not found"
}
2021/12/23 14:04:09 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:04:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:04:16 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:04:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:04:21 Deploying app: TestCancelTimerBucketop. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:04:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop stored in temp store"
}
2021/12/23 14:04:27 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/23 14:04:32 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/23 14:04:37 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/23 14:04:42 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/23 14:04:42 App: TestCancelTimerBucketop got deployed
2021/12/23 14:04:42 Starting bucket ops to source bucket
2021/12/23 14:04:50 Starting bucket ops to source bucket
2021/12/23 14:05:52 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/23 14:05:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:05:57 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/23 14:06:02 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/23 14:06:07 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/23 14:06:12 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/23 14:06:12 App: TestCancelTimerBucketop got un-deployed
2021/12/23 14:06:12 No eventing-con process running
2021/12/23 14:06:12 Function: TestCancelTimerBucketop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop deleting in background"
}
2021/12/23 14:06:14 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:06:24 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:06:29 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/23 14:06:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:06:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:06:41 Waiting for dst bucket item count to get to: 0 curr count: 882
2021/12/23 14:06:46 Waiting for dst bucket item count to get to: 0 curr count: 882
2021/12/23 14:06:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCancelTimerBucketop (163.34s)
=== RUN   TestJSExpiryDate
2021/12/23 14:06:51 Starting bucket ops to source bucket
2021/12/23 14:07:04 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 14:07:04 Deploying app: TestJSExpiryDate. Item count src bucket: 5000 dst bucket: 0
2021/12/23 14:07:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate stored in temp store"
}
2021/12/23 14:07:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:07:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:07:40 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/23 14:07:45 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/23 14:07:50 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/23 14:07:55 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/23 14:07:55 App: TestJSExpiryDate got un-deployed
2021/12/23 14:07:55 No eventing-con process running
2021/12/23 14:07:55 Function: TestJSExpiryDate delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate deleting in background"
}
2021/12/23 14:07:57 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:08:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:08:04 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:08:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestJSExpiryDate (83.61s)
=== RUN   TestBinaryDoc
2021/12/23 14:08:14 Starting bucket ops to source bucket
2021/12/23 14:08:14 Deploying app: TestBinaryDoc. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:08:16 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc stored in temp store"
}
2021/12/23 14:08:16 Starting bucket ops to source bucket
2021/12/23 14:08:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:08:16 Deleting source bucket docs now
2021/12/23 14:08:16 Starting bucket ops to source bucket
2021/12/23 14:08:53 Waiting for dst bucket item count to get to: 32 curr count: 21
2021/12/23 14:08:58 Waiting for dst bucket item count to get to: 32 curr count: 31
2021/12/23 14:09:03 Waiting for dst bucket item count to get to: 32 curr count: 31
2021/12/23 14:09:03 src & dst bucket item count matched up. src bucket count: 32 dst bucket count: 32
2021/12/23 14:09:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:09:08 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/23 14:09:13 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/23 14:09:18 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/23 14:09:18 App: TestBinaryDoc got un-deployed
2021/12/23 14:09:18 No eventing-con process running
2021/12/23 14:09:18 Function: TestBinaryDoc delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc deleting in background"
}
2021/12/23 14:09:20 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:09:30 Waiting for dst bucket item count to get to: 0 curr count: 17
2021/12/23 14:09:35 Waiting for dst bucket item count to get to: 0 curr count: 17
2021/12/23 14:09:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:09:37 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:09:47 Waiting for dst bucket item count to get to: 0 curr count: 32
2021/12/23 14:09:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBinaryDoc (97.39s)
=== RUN   TestConstantBindings
2021/12/23 14:09:47 Deploying app: TestConstantBindings. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:09:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings stored in temp store"
}
2021/12/23 14:09:48 Starting bucket ops to source bucket
2021/12/23 14:09:54 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/23 14:09:59 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/23 14:10:04 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/23 14:10:09 Waiting for dst bucket item count to get to: 6000 curr count: 48
2021/12/23 14:10:14 Waiting for dst bucket item count to get to: 6000 curr count: 48
2021/12/23 14:10:14 src & dst bucket item count matched up. src bucket count: 6000 dst bucket count: 6000
2021/12/23 14:10:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:10:19 Waiting for app: TestConstantBindings to get un-deployed
2021/12/23 14:10:24 Waiting for app: TestConstantBindings to get un-deployed
2021/12/23 14:10:29 Waiting for app: TestConstantBindings to get un-deployed
2021/12/23 14:10:34 Waiting for app: TestConstantBindings to get un-deployed
2021/12/23 14:10:34 App: TestConstantBindings got un-deployed
2021/12/23 14:10:34 No eventing-con process running
2021/12/23 14:10:34 Function: TestConstantBindings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings deleting in background"
}
2021/12/23 14:10:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:10:46 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:10:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:10:47 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:10:57 Waiting for dst bucket item count to get to: 0 curr count: 3000
2021/12/23 14:10:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestConstantBindings (70.75s)
=== RUN   TestCodeInsight
2021/12/23 14:11:02 Waiting for app: TestCodeInsight to get un-deployed
2021/12/23 14:11:03 App: TestCodeInsight got un-deployed
2021/12/23 14:11:03 No eventing-con process running
2021/12/23 14:11:03 Function: TestCodeInsight delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCodeInsight not found"
}
2021/12/23 14:11:04 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:11:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:11:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:11:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:11:16 Starting bucket ops to source bucket
2021/12/23 14:11:17 Deploying app: TestCodeInsight. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:11:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight stored in temp store"
}
2021/12/23 14:11:23 Waiting for app: TestCodeInsight to get deployed
2021/12/23 14:11:28 Waiting for app: TestCodeInsight to get deployed
2021/12/23 14:11:33 Waiting for app: TestCodeInsight to get deployed
2021/12/23 14:11:38 Waiting for app: TestCodeInsight to get deployed
2021/12/23 14:11:38 App: TestCodeInsight got deployed
2021/12/23 14:11:43 Waiting for dst bucket item count to get to: 1000 curr count: 23
2021/12/23 14:11:43 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/23 14:11:48 Waiting for app: TestCodeInsight to get un-deployed
2021/12/23 14:11:53 Waiting for app: TestCodeInsight to get un-deployed
2021/12/23 14:11:58 Waiting for app: TestCodeInsight to get un-deployed
2021/12/23 14:11:58 App: TestCodeInsight got un-deployed
2021/12/23 14:11:58 No eventing-con process running
2021/12/23 14:11:59 Function: TestCodeInsight delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight deleting in background"
}
2021/12/23 14:12:00 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:12:10 Waiting for dst bucket item count to get to: 0 curr count: 724
2021/12/23 14:12:15 Waiting for dst bucket item count to get to: 0 curr count: 724
2021/12/23 14:12:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:12:17 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:12:27 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:12:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCodeInsight (89.63s)
=== RUN   TestParserTransform
--- PASS: TestParserTransform (0.04s)
=== RUN   TestParserDetect
--- PASS: TestParserDetect (0.00s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	12858.524s

Functional tests took 3 hours, 34 minutes and 22 seconds.

N1ql tests

2021/12/23 14:12:37 Started n1ql
2021/12/23 14:12:47 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/23 14:12:47 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/23 14:12:47 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 14:12:47 Removing node(s): [] from the cluster
2021/12/23 14:12:52 Rebalance progress: 100
=== RUN   TestRecursiveMutationN1QL
2021/12/23 14:12:57 Waiting for app: TestRecursiveMutationN1QL to get un-deployed
2021/12/23 14:12:57 App: TestRecursiveMutationN1QL got un-deployed
2021/12/23 14:12:57 No eventing-con process running
2021/12/23 14:12:57 Function: TestRecursiveMutationN1QL delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestRecursiveMutationN1QL not found"
}
2021/12/23 14:12:59 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:13:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:13:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:13:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:13:11 Deploying app: TestRecursiveMutationN1QL. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:13:11 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Function: TestRecursiveMutationN1QL N1QL DML to source keyspace default:default"
}
--- PASS: TestRecursiveMutationN1QL (18.63s)
=== RUN   TestFlexReset1
2021/12/23 14:13:16 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/23 14:13:16 App: TestFlexReset1 got un-deployed
2021/12/23 14:13:16 No eventing-con process running
2021/12/23 14:13:16 Function: TestFlexReset1 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset1 not found"
}
2021/12/23 14:13:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:13:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:13:24 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:13:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:13:29 Starting bucket ops to source bucket
2021/12/23 14:13:30 Deploying app: TestFlexReset1. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:13:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 stored in temp store"
}
2021/12/23 14:13:36 Waiting for app: TestFlexReset1 to get deployed
2021/12/23 14:13:41 Waiting for app: TestFlexReset1 to get deployed
2021/12/23 14:13:46 Waiting for app: TestFlexReset1 to get deployed
2021/12/23 14:13:51 Waiting for app: TestFlexReset1 to get deployed
2021/12/23 14:13:51 App: TestFlexReset1 got deployed
2021/12/23 14:13:51 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/23 14:13:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:13:57 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/23 14:14:02 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/23 14:14:07 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/23 14:14:12 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/23 14:14:12 App: TestFlexReset1 got un-deployed
2021/12/23 14:14:12 No eventing-con process running
2021/12/23 14:14:12 Function: TestFlexReset1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 deleting in background"
}
2021/12/23 14:14:13 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:14:23 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:14:28 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/23 14:14:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:14:30 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:14:40 Waiting for dst bucket item count to get to: 0 curr count: 758
2021/12/23 14:14:45 Waiting for dst bucket item count to get to: 0 curr count: 758
2021/12/23 14:14:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset1 (94.42s)
=== RUN   TestFlexReset2
2021/12/23 14:14:50 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/23 14:14:50 App: TestFlexReset2 got un-deployed
2021/12/23 14:14:50 No eventing-con process running
2021/12/23 14:14:50 Function: TestFlexReset2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset2 not found"
}
2021/12/23 14:14:52 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:14:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:14:59 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:15:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:15:04 Starting bucket ops to source bucket
2021/12/23 14:15:04 Deploying app: TestFlexReset2. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:15:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 stored in temp store"
}
2021/12/23 14:15:11 Waiting for app: TestFlexReset2 to get deployed
2021/12/23 14:15:16 Waiting for app: TestFlexReset2 to get deployed
2021/12/23 14:15:21 Waiting for app: TestFlexReset2 to get deployed
2021/12/23 14:15:26 Waiting for app: TestFlexReset2 to get deployed
2021/12/23 14:15:26 App: TestFlexReset2 got deployed
2021/12/23 14:15:31 Waiting for dst bucket item count to get to: 1000 curr count: 315
2021/12/23 14:15:36 Waiting for dst bucket item count to get to: 1000 curr count: 815
2021/12/23 14:15:36 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/23 14:15:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:15:41 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/23 14:15:46 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/23 14:15:51 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/23 14:15:56 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/23 14:15:56 App: TestFlexReset2 got un-deployed
2021/12/23 14:15:56 No eventing-con process running
2021/12/23 14:15:56 Function: TestFlexReset2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 deleting in background"
}
2021/12/23 14:15:58 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:16:08 Waiting for dst bucket item count to get to: 0 curr count: 548
2021/12/23 14:16:13 Waiting for dst bucket item count to get to: 0 curr count: 48
2021/12/23 14:16:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:16:15 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:16:25 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/23 14:16:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset2 (99.24s)
=== RUN   TestN1QLLabelledBreak
2021/12/23 14:16:30 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/23 14:16:30 App: TestN1QLLabelledBreak got un-deployed
2021/12/23 14:16:30 No eventing-con process running
2021/12/23 14:16:30 Function: TestN1QLLabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLLabelledBreak not found"
}
2021/12/23 14:16:31 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:16:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:16:38 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:16:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:16:43 Deploying app: TestN1QLLabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:16:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak stored in temp store"
}
2021/12/23 14:16:49 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/23 14:16:54 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/23 14:16:59 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/23 14:17:04 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/23 14:17:04 App: TestN1QLLabelledBreak got deployed
2021/12/23 14:17:04 Starting bucket ops to source bucket
2021/12/23 14:17:14 Waiting for dst bucket item count to get to: 10000 curr count: 193
2021/12/23 14:17:19 Waiting for dst bucket item count to get to: 10000 curr count: 193
2021/12/23 14:17:24 Waiting for dst bucket item count to get to: 10000 curr count: 596
2021/12/23 14:17:29 Waiting for dst bucket item count to get to: 10000 curr count: 2378
2021/12/23 14:17:34 Waiting for dst bucket item count to get to: 10000 curr count: 4389
2021/12/23 14:17:39 Waiting for dst bucket item count to get to: 10000 curr count: 6416
2021/12/23 14:17:44 Waiting for dst bucket item count to get to: 10000 curr count: 8416
2021/12/23 14:17:49 Waiting for dst bucket item count to get to: 10000 curr count: 9414
2021/12/23 14:17:49 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 14:17:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:17:54 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/23 14:17:59 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/23 14:18:04 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/23 14:18:09 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/23 14:18:09 App: TestN1QLLabelledBreak got un-deployed
2021/12/23 14:18:09 No eventing-con process running
2021/12/23 14:18:09 Function: TestN1QLLabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak deleting in background"
}
2021/12/23 14:18:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:18:21 Waiting for dst bucket item count to get to: 0 curr count: 4929
2021/12/23 14:18:26 Waiting for dst bucket item count to get to: 0 curr count: 4929
2021/12/23 14:18:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:18:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:18:38 Waiting for dst bucket item count to get to: 0 curr count: 6070
2021/12/23 14:18:43 Waiting for dst bucket item count to get to: 0 curr count: 1070
2021/12/23 14:18:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLLabelledBreak (138.25s)
=== RUN   TestN1QLUnlabelledBreak
2021/12/23 14:18:48 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/23 14:18:48 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/23 14:18:48 No eventing-con process running
2021/12/23 14:18:48 Function: TestN1QLUnlabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLUnlabelledBreak not found"
}
2021/12/23 14:18:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:18:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:18:56 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:19:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:19:01 Deploying app: TestN1QLUnlabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:19:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak stored in temp store"
}
2021/12/23 14:19:07 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/23 14:19:12 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/23 14:19:17 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/23 14:19:23 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/23 14:19:23 App: TestN1QLUnlabelledBreak got deployed
2021/12/23 14:19:23 Starting bucket ops to source bucket
2021/12/23 14:19:32 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/23 14:19:37 Waiting for dst bucket item count to get to: 10000 curr count: 1596
2021/12/23 14:19:42 Waiting for dst bucket item count to get to: 10000 curr count: 1596
2021/12/23 14:19:47 Waiting for dst bucket item count to get to: 10000 curr count: 1596
2021/12/23 14:19:52 Waiting for dst bucket item count to get to: 10000 curr count: 7338
2021/12/23 14:19:57 Waiting for dst bucket item count to get to: 10000 curr count: 7338
2021/12/23 14:19:57 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 14:19:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:20:02 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/23 14:20:07 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/23 14:20:12 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/23 14:20:17 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/23 14:20:17 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/23 14:20:17 No eventing-con process running
2021/12/23 14:20:18 Function: TestN1QLUnlabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak deleting in background"
}
2021/12/23 14:20:19 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:20:29 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 14:20:34 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 14:20:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:20:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:20:46 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 14:20:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLUnlabelledBreak (123.36s)
=== RUN   TestN1QLThrowStatement
2021/12/23 14:20:51 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/23 14:20:51 App: TestN1QLThrowStatement got un-deployed
2021/12/23 14:20:51 No eventing-con process running
2021/12/23 14:20:51 Function: TestN1QLThrowStatement delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLThrowStatement not found"
}
2021/12/23 14:20:53 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:20:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:21:00 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:21:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:21:05 Deploying app: TestN1QLThrowStatement. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:21:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement stored in temp store"
}
2021/12/23 14:21:11 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/23 14:21:16 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/23 14:21:21 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/23 14:21:26 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/23 14:21:26 App: TestN1QLThrowStatement got deployed
2021/12/23 14:21:26 Starting bucket ops to source bucket
2021/12/23 14:21:36 Waiting for dst bucket item count to get to: 10000 curr count: 305
2021/12/23 14:21:41 Waiting for dst bucket item count to get to: 10000 curr count: 305
2021/12/23 14:21:46 Waiting for dst bucket item count to get to: 10000 curr count: 3686
2021/12/23 14:21:51 Waiting for dst bucket item count to get to: 10000 curr count: 3686
2021/12/23 14:21:56 Waiting for dst bucket item count to get to: 10000 curr count: 3686
2021/12/23 14:22:01 Waiting for dst bucket item count to get to: 10000 curr count: 7639
2021/12/23 14:22:06 Waiting for dst bucket item count to get to: 10000 curr count: 7639
2021/12/23 14:22:06 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/23 14:22:06 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:22:11 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/23 14:22:16 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/23 14:22:21 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/23 14:22:26 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/23 14:22:26 App: TestN1QLThrowStatement got un-deployed
2021/12/23 14:22:26 No eventing-con process running
2021/12/23 14:22:26 Function: TestN1QLThrowStatement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement deleting in background"
}
2021/12/23 14:22:28 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:22:38 Waiting for dst bucket item count to get to: 0 curr count: 3200
2021/12/23 14:22:43 Waiting for dst bucket item count to get to: 0 curr count: 700
2021/12/23 14:22:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:22:45 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:22:55 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/23 14:22:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLThrowStatement (128.46s)
=== RUN   TestN1QLNestedForLoop
2021/12/23 14:23:00 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/23 14:23:00 App: TestN1QLNestedForLoop got un-deployed
2021/12/23 14:23:00 No eventing-con process running
2021/12/23 14:23:00 Function: TestN1QLNestedForLoop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLNestedForLoop not found"
}
2021/12/23 14:23:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:23:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:23:08 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:23:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:23:13 Deploying app: TestN1QLNestedForLoop. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:23:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop stored in temp store"
}
2021/12/23 14:23:19 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/23 14:23:24 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/23 14:23:29 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/23 14:23:34 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/23 14:23:34 App: TestN1QLNestedForLoop got deployed
2021/12/23 14:23:34 Starting bucket ops to source bucket
2021/12/23 14:23:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/23 14:23:49 Waiting for dst bucket item count to get to: 5000 curr count: 96
2021/12/23 14:23:54 Waiting for dst bucket item count to get to: 5000 curr count: 323
2021/12/23 14:23:59 Waiting for dst bucket item count to get to: 5000 curr count: 1386
2021/12/23 14:24:04 Waiting for dst bucket item count to get to: 5000 curr count: 2576
2021/12/23 14:24:09 Waiting for dst bucket item count to get to: 5000 curr count: 3797
2021/12/23 14:24:14 Waiting for dst bucket item count to get to: 5000 curr count: 4880
2021/12/23 14:24:14 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 14:24:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:24:19 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/23 14:24:24 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/23 14:24:29 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/23 14:24:34 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/23 14:24:34 App: TestN1QLNestedForLoop got un-deployed
2021/12/23 14:24:34 No eventing-con process running
2021/12/23 14:24:34 Function: TestN1QLNestedForLoop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop deleting in background"
}
2021/12/23 14:24:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:24:46 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 14:24:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:24:47 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:24:57 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 14:24:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLNestedForLoop (122.89s)
=== RUN   TestN1QLAutoClose
2021/12/23 14:25:02 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/23 14:25:02 App: TestN1QLAutoClose got un-deployed
2021/12/23 14:25:03 No eventing-con process running
2021/12/23 14:25:03 Function: TestN1QLAutoClose delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLAutoClose not found"
}
2021/12/23 14:25:04 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:25:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:25:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:25:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:25:16 Deploying app: TestN1QLAutoClose. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:25:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose stored in temp store"
}
2021/12/23 14:25:22 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/23 14:25:27 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/23 14:25:32 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/23 14:25:37 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/23 14:25:37 App: TestN1QLAutoClose got deployed
2021/12/23 14:25:37 Starting bucket ops to source bucket
2021/12/23 14:25:43 Waiting for dst bucket item count to get to: 500 curr count: 0
2021/12/23 14:25:48 Waiting for dst bucket item count to get to: 500 curr count: 1
2021/12/23 14:25:53 Waiting for dst bucket item count to get to: 500 curr count: 1
2021/12/23 14:25:58 Waiting for dst bucket item count to get to: 500 curr count: 22
2021/12/23 14:26:03 Waiting for dst bucket item count to get to: 500 curr count: 22
2021/12/23 14:26:08 Waiting for dst bucket item count to get to: 500 curr count: 44
2021/12/23 14:26:13 Waiting for dst bucket item count to get to: 500 curr count: 44
2021/12/23 14:26:18 Waiting for dst bucket item count to get to: 500 curr count: 66
2021/12/23 14:26:23 Waiting for dst bucket item count to get to: 500 curr count: 66
2021/12/23 14:26:28 Waiting for dst bucket item count to get to: 500 curr count: 88
2021/12/23 14:26:33 Waiting for dst bucket item count to get to: 500 curr count: 97
2021/12/23 14:26:38 Waiting for dst bucket item count to get to: 500 curr count: 108
2021/12/23 14:26:43 Waiting for dst bucket item count to get to: 500 curr count: 118
2021/12/23 14:26:48 Waiting for dst bucket item count to get to: 500 curr count: 130
2021/12/23 14:26:53 Waiting for dst bucket item count to get to: 500 curr count: 140
2021/12/23 14:26:58 Waiting for dst bucket item count to get to: 500 curr count: 140
2021/12/23 14:27:03 Waiting for dst bucket item count to get to: 500 curr count: 153
2021/12/23 14:27:08 Waiting for dst bucket item count to get to: 500 curr count: 163
2021/12/23 14:27:13 Waiting for dst bucket item count to get to: 500 curr count: 174
2021/12/23 14:27:18 Waiting for dst bucket item count to get to: 500 curr count: 185
2021/12/23 14:27:23 Waiting for dst bucket item count to get to: 500 curr count: 196
2021/12/23 14:27:28 Waiting for dst bucket item count to get to: 500 curr count: 207
2021/12/23 14:27:33 Waiting for dst bucket item count to get to: 500 curr count: 218
2021/12/23 14:27:38 Waiting for dst bucket item count to get to: 500 curr count: 227
2021/12/23 14:27:43 Waiting for dst bucket item count to get to: 500 curr count: 239
2021/12/23 14:27:48 Waiting for dst bucket item count to get to: 500 curr count: 249
2021/12/23 14:27:53 Waiting for dst bucket item count to get to: 500 curr count: 259
2021/12/23 14:27:58 Waiting for dst bucket item count to get to: 500 curr count: 268
2021/12/23 14:28:03 Waiting for dst bucket item count to get to: 500 curr count: 281
2021/12/23 14:28:08 Waiting for dst bucket item count to get to: 500 curr count: 303
2021/12/23 14:28:13 Waiting for dst bucket item count to get to: 500 curr count: 303
2021/12/23 14:28:18 Waiting for dst bucket item count to get to: 500 curr count: 324
2021/12/23 14:28:23 Waiting for dst bucket item count to get to: 500 curr count: 324
2021/12/23 14:28:28 Waiting for dst bucket item count to get to: 500 curr count: 346
2021/12/23 14:28:33 Waiting for dst bucket item count to get to: 500 curr count: 346
2021/12/23 14:28:38 Waiting for dst bucket item count to get to: 500 curr count: 366
2021/12/23 14:28:43 Waiting for dst bucket item count to get to: 500 curr count: 366
2021/12/23 14:28:48 Waiting for dst bucket item count to get to: 500 curr count: 387
2021/12/23 14:28:53 Waiting for dst bucket item count to get to: 500 curr count: 387
2021/12/23 14:28:58 Waiting for dst bucket item count to get to: 500 curr count: 409
2021/12/23 14:29:03 Waiting for dst bucket item count to get to: 500 curr count: 409
2021/12/23 14:29:08 Waiting for dst bucket item count to get to: 500 curr count: 430
2021/12/23 14:29:13 Waiting for dst bucket item count to get to: 500 curr count: 430
2021/12/23 14:29:18 Waiting for dst bucket item count to get to: 500 curr count: 453
2021/12/23 14:29:24 Waiting for dst bucket item count to get to: 500 curr count: 453
2021/12/23 14:29:29 Waiting for dst bucket item count to get to: 500 curr count: 475
2021/12/23 14:29:34 Waiting for dst bucket item count to get to: 500 curr count: 475
2021/12/23 14:29:39 Waiting for dst bucket item count to get to: 500 curr count: 494
2021/12/23 14:29:44 Waiting for dst bucket item count to get to: 500 curr count: 494
2021/12/23 14:29:44 src & dst bucket item count matched up. src bucket count: 500 dst bucket count: 500
2021/12/23 14:29:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:29:49 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/23 14:29:54 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/23 14:29:59 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/23 14:30:04 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/23 14:30:04 App: TestN1QLAutoClose got un-deployed
2021/12/23 14:30:04 No eventing-con process running
2021/12/23 14:30:04 Function: TestN1QLAutoClose delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose deleting in background"
}
2021/12/23 14:30:06 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:30:16 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/23 14:30:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:30:17 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:30:28 Waiting for dst bucket item count to get to: 0 curr count: 250
2021/12/23 14:30:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLAutoClose (330.03s)
=== RUN   TestN1QLPosParams
2021/12/23 14:30:33 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/23 14:30:33 App: TestN1QLPosParams got un-deployed
2021/12/23 14:30:33 No eventing-con process running
2021/12/23 14:30:33 Function: TestN1QLPosParams delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLPosParams not found"
}
2021/12/23 14:30:34 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:30:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:30:41 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:30:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:30:46 Deploying app: TestN1QLPosParams. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:30:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams stored in temp store"
}
2021/12/23 14:30:52 Waiting for app: TestN1QLPosParams to get deployed
2021/12/23 14:30:57 Waiting for app: TestN1QLPosParams to get deployed
2021/12/23 14:31:02 Waiting for app: TestN1QLPosParams to get deployed
2021/12/23 14:31:07 Waiting for app: TestN1QLPosParams to get deployed
2021/12/23 14:31:07 App: TestN1QLPosParams got deployed
2021/12/23 14:31:07 Starting bucket ops to source bucket
2021/12/23 14:31:17 Waiting for dst bucket item count to get to: 5000 curr count: 316
2021/12/23 14:31:22 Waiting for dst bucket item count to get to: 5000 curr count: 316
2021/12/23 14:31:22 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 14:31:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:31:27 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/23 14:31:32 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/23 14:31:37 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/23 14:31:42 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/23 14:31:42 App: TestN1QLPosParams got un-deployed
2021/12/23 14:31:42 No eventing-con process running
2021/12/23 14:31:42 Function: TestN1QLPosParams delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams deleting in background"
}
2021/12/23 14:31:44 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:31:54 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 14:31:59 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/23 14:31:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:32:01 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:32:11 Waiting for dst bucket item count to get to: 0 curr count: 4823
2021/12/23 14:32:16 Waiting for dst bucket item count to get to: 0 curr count: 4823
2021/12/23 14:32:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLPosParams (108.38s)
=== RUN   TestN1QLExhaustConnPool
2021/12/23 14:32:21 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/23 14:32:21 App: TestN1QLExhaustConnPool got un-deployed
2021/12/23 14:32:21 No eventing-con process running
2021/12/23 14:32:21 Function: TestN1QLExhaustConnPool delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLExhaustConnPool not found"
}
2021/12/23 14:32:23 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:32:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:32:29 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:32:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:32:34 Deploying app: TestN1QLExhaustConnPool. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:32:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool stored in temp store"
}
2021/12/23 14:32:41 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/23 14:32:46 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/23 14:32:51 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/23 14:32:56 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/23 14:32:56 App: TestN1QLExhaustConnPool got deployed
2021/12/23 14:32:56 Starting bucket ops to source bucket
2021/12/23 14:33:01 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/23 14:33:06 Waiting for dst bucket item count to get to: 100 curr count: 70
2021/12/23 14:33:11 Waiting for dst bucket item count to get to: 100 curr count: 70
2021/12/23 14:33:11 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/23 14:33:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:33:16 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/23 14:33:21 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/23 14:33:26 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/23 14:33:31 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/23 14:33:31 App: TestN1QLExhaustConnPool got un-deployed
2021/12/23 14:33:31 No eventing-con process running
2021/12/23 14:33:31 Function: TestN1QLExhaustConnPool delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool deleting in background"
}
2021/12/23 14:33:33 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:33:43 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/23 14:33:48 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 14:33:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:33:50 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:34:00 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 14:34:05 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/23 14:34:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLExhaustConnPool (108.72s)
=== RUN   TestN1QLGraceSufficientTimeOut
2021/12/23 14:34:10 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/23 14:34:10 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/23 14:34:10 No eventing-con process running
2021/12/23 14:34:10 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLGraceSufficientTimeOut not found"
}
2021/12/23 14:34:11 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:34:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:34:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:34:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:34:23 Starting bucket ops to source bucket
2021/12/23 14:34:40 Starting bucket ops to source bucket
2021/12/23 14:34:58 Deploying app: TestN1QLGraceSufficientTimeOut. Item count src bucket: 30000 dst bucket: 13858
2021/12/23 14:34:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut stored in temp store"
}
2021/12/23 14:35:04 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/23 14:35:09 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/23 14:35:14 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/23 14:35:19 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/23 14:35:19 App: TestN1QLGraceSufficientTimeOut got deployed
2021/12/23 14:35:19 Starting bucket ops to source bucket
2021/12/23 14:35:24 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:29 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:34 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:39 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:44 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:49 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:54 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:35:59 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:36:04 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:36:09 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:36:14 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/23 14:36:14 src & dst bucket item count matched up. src bucket count: 30001 dst bucket count: 30001
2021/12/23 14:36:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:36:19 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/23 14:36:24 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/23 14:36:29 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/23 14:36:34 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/23 14:36:34 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/23 14:36:34 No eventing-con process running
2021/12/23 14:36:34 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut deleting in background"
}
2021/12/23 14:36:36 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:36:46 Waiting for dst bucket item count to get to: 0 curr count: 30000
2021/12/23 14:36:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:36:48 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:36:58 Waiting for dst bucket item count to get to: 0 curr count: 19504
2021/12/23 14:37:03 Waiting for dst bucket item count to get to: 0 curr count: 4504
2021/12/23 14:37:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLGraceSufficientTimeOut (178.33s)
=== RUN   TestN1QLQueryConsistency
2021/12/23 14:37:08 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/23 14:37:08 App: n1qlquery_consistency got un-deployed
2021/12/23 14:37:08 No eventing-con process running
2021/12/23 14:37:08 Function: n1qlquery_consistency delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: n1qlquery_consistency not found"
}
2021/12/23 14:37:10 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:37:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:37:16 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:37:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:37:21 Starting bucket ops to source bucket
2021/12/23 14:37:25 Deploying app: n1qlquery_consistency. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:37:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency stored in temp store"
}
2021/12/23 14:37:31 Waiting for app: n1qlquery_consistency to get deployed
2021/12/23 14:37:36 Waiting for app: n1qlquery_consistency to get deployed
2021/12/23 14:37:41 Waiting for app: n1qlquery_consistency to get deployed
2021/12/23 14:37:46 Waiting for app: n1qlquery_consistency to get deployed
2021/12/23 14:37:46 App: n1qlquery_consistency got deployed
2021/12/23 14:37:51 Waiting for dst bucket item count to get to: 5000 curr count: 327
2021/12/23 14:37:56 Waiting for dst bucket item count to get to: 5000 curr count: 327
2021/12/23 14:37:56 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/23 14:37:56 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/23 14:38:01 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/23 14:38:06 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/23 14:38:11 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/23 14:38:16 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/23 14:38:16 App: n1qlquery_consistency got un-deployed
2021/12/23 14:38:16 No eventing-con process running
2021/12/23 14:38:16 Function: n1qlquery_consistency delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency deleting in background"
}
2021/12/23 14:38:18 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:38:28 Waiting for dst bucket item count to get to: 0 curr count: 3217
2021/12/23 14:38:33 Waiting for dst bucket item count to get to: 0 curr count: 717
2021/12/23 14:38:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:38:35 Context: Bucket flush, response code: 200 dump: 
2021/12/23 14:38:45 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/23 14:38:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLQueryConsistency (101.88s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	1563.526s

N1ql tests took 0 hours, 26 minutes and 8 seconds.

Starting server
Starting server: attempt 1

Rebalance tests

2021/12/23 14:40:20 Started rebalance
Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:56818->127.0.0.1:9000: read: connection reset by peer
Quota setup Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:56818->127.0.0.1:9000: read: connection reset by peer
=== RUN   TestDuplicateWithEventingRebIn
2021/12/23 14:40:39 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/23 14:40:39 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/23 14:40:39 No eventing-con process running
2021/12/23 14:40:39 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebIn not found"
}
2021/12/23 14:40:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:40:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:40:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:40:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:40:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:40:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:40:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:40:57 Deploying app: TestDuplicateWithEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:40:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn stored in temp store"
}
2021/12/23 14:41:03 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/23 14:41:08 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/23 14:41:13 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/23 14:41:18 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/23 14:41:18 App: TestDuplicateWithEventingRebIn got deployed
2021/12/23 14:41:18 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 14:41:18 Starting bucket ops to source bucket
2021/12/23 14:41:33 Removing node(s): [] from the cluster
2021/12/23 14:41:38 Rebalance progress: 0
2021/12/23 14:41:43 Rebalance progress: 0
2021/12/23 14:41:48 Rebalance progress: 25
2021/12/23 14:41:53 Rebalance progress: 25
2021/12/23 14:41:58 Rebalance progress: 25
2021/12/23 14:42:03 Rebalance progress: 25
2021/12/23 14:42:08 Rebalance progress: 25
2021/12/23 14:42:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:42:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:42:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:42:13 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/23 14:42:13 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:42:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 14:42:18 Rebalance progress: 25
2021/12/23 14:42:23 Rebalance progress: 25
2021/12/23 14:42:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:42:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:42:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:42:28 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/23 14:42:28 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:42:28 Waiting for src bucket item count to get to: 200000 curr count: 100747
2021/12/23 14:42:33 Waiting for src bucket item count to get to: 200000 curr count: 100747
2021/12/23 14:42:38 Waiting for src bucket item count to get to: 200000 curr count: 117435
2021/12/23 14:42:43 Waiting for src bucket item count to get to: 200000 curr count: 117435
2021/12/23 14:42:48 Waiting for src bucket item count to get to: 200000 curr count: 134201
2021/12/23 14:42:53 Waiting for src bucket item count to get to: 200000 curr count: 134201
2021/12/23 14:42:58 Waiting for src bucket item count to get to: 200000 curr count: 151005
2021/12/23 14:43:03 Waiting for src bucket item count to get to: 200000 curr count: 151005
2021/12/23 14:43:08 Waiting for src bucket item count to get to: 200000 curr count: 167585
2021/12/23 14:43:13 Waiting for src bucket item count to get to: 200000 curr count: 167585
2021/12/23 14:43:18 Waiting for src bucket item count to get to: 200000 curr count: 184327
2021/12/23 14:43:23 Waiting for src bucket item count to get to: 200000 curr count: 184327
2021/12/23 14:43:28 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 14:43:28 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 14:43:33 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/23 14:43:38 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/23 14:43:43 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/23 14:43:48 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/23 14:43:48 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/23 14:43:48 No eventing-con process running
2021/12/23 14:43:49 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn deleting in background"
}
2021/12/23 14:43:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:43:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:44:00 Waiting for dst bucket item count to get to: 0 curr count: 66428
2021/12/23 14:44:05 Waiting for dst bucket item count to get to: 0 curr count: 66428
2021/12/23 14:44:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:44:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:44:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:44:17 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:44:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebIn (228.42s)
=== RUN   TestDuplicateWithEventingRebOut
2021/12/23 14:44:22 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 14:44:22 Removing node(s): [] from the cluster
2021/12/23 14:44:27 Rebalance progress: 100
2021/12/23 14:44:32 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/23 14:44:32 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/23 14:44:32 No eventing-con process running
2021/12/23 14:44:32 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebOut not found"
}
2021/12/23 14:44:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:44:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:44:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:44:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:44:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:44:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:44:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:44:51 Deploying app: TestDuplicateWithEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:44:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut stored in temp store"
}
2021/12/23 14:44:57 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/23 14:45:02 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/23 14:45:07 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/23 14:45:12 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/23 14:45:12 App: TestDuplicateWithEventingRebOut got deployed
2021/12/23 14:45:12 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 14:45:12 Starting bucket ops to source bucket
2021/12/23 14:45:17 Rebalance progress: 25
2021/12/23 14:45:22 Rebalance progress: 25
2021/12/23 14:45:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:45:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:45:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:45:27 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/23 14:45:27 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:45:27 Waiting for src bucket item count to get to: 200000 curr count: 12065
2021/12/23 14:45:32 Waiting for src bucket item count to get to: 200000 curr count: 28792
2021/12/23 14:45:37 Waiting for src bucket item count to get to: 200000 curr count: 28792
2021/12/23 14:45:42 Waiting for src bucket item count to get to: 200000 curr count: 45533
2021/12/23 14:45:47 Waiting for src bucket item count to get to: 200000 curr count: 45533
2021/12/23 14:45:52 Waiting for src bucket item count to get to: 200000 curr count: 62441
2021/12/23 14:45:57 Waiting for src bucket item count to get to: 200000 curr count: 62441
2021/12/23 14:46:02 Waiting for src bucket item count to get to: 200000 curr count: 62441
2021/12/23 14:46:07 Waiting for src bucket item count to get to: 200000 curr count: 78959
2021/12/23 14:46:12 Waiting for src bucket item count to get to: 200000 curr count: 78959
2021/12/23 14:46:17 Waiting for src bucket item count to get to: 200000 curr count: 95548
2021/12/23 14:46:22 Waiting for src bucket item count to get to: 200000 curr count: 95548
2021/12/23 14:46:27 Waiting for src bucket item count to get to: 200000 curr count: 112474
2021/12/23 14:46:32 Waiting for src bucket item count to get to: 200000 curr count: 112474
2021/12/23 14:46:37 Waiting for src bucket item count to get to: 200000 curr count: 129419
2021/12/23 14:46:42 Waiting for src bucket item count to get to: 200000 curr count: 129419
2021/12/23 14:46:47 Waiting for src bucket item count to get to: 200000 curr count: 146215
2021/12/23 14:46:52 Waiting for src bucket item count to get to: 200000 curr count: 146215
2021/12/23 14:46:57 Waiting for src bucket item count to get to: 200000 curr count: 162863
2021/12/23 14:47:02 Waiting for src bucket item count to get to: 200000 curr count: 162863
2021/12/23 14:47:07 Waiting for src bucket item count to get to: 200000 curr count: 179412
2021/12/23 14:47:12 Waiting for src bucket item count to get to: 200000 curr count: 179412
2021/12/23 14:47:17 Waiting for src bucket item count to get to: 200000 curr count: 195752
2021/12/23 14:47:22 Waiting for src bucket item count to get to: 200000 curr count: 195752
2021/12/23 14:47:27 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 14:47:27 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 14:47:32 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/23 14:47:37 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/23 14:47:42 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/23 14:47:47 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/23 14:47:47 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/23 14:47:47 No eventing-con process running
2021/12/23 14:47:47 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut deleting in background"
}
2021/12/23 14:47:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:47:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:47:59 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:47:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:48:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:48:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:48:11 Waiting for dst bucket item count to get to: 0 curr count: 142136
2021/12/23 14:48:16 Waiting for dst bucket item count to get to: 0 curr count: 142136
2021/12/23 14:48:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebOut (238.65s)
=== RUN   TestDuplicateWithEventingSwapReb
2021/12/23 14:48:26 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/23 14:48:26 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/23 14:48:26 No eventing-con process running
2021/12/23 14:48:26 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingSwapReb not found"
}
2021/12/23 14:48:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:48:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:48:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:48:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:48:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:48:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:48:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:48:44 Deploying app: TestDuplicateWithEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:48:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb stored in temp store"
}
2021/12/23 14:48:50 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/23 14:48:55 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/23 14:49:00 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/23 14:49:05 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/23 14:49:05 App: TestDuplicateWithEventingSwapReb got deployed
2021/12/23 14:49:05 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 14:49:05 Starting bucket ops to source bucket
2021/12/23 14:49:06 Removing node(s): [] from the cluster
2021/12/23 14:49:11 Rebalance progress: 25
2021/12/23 14:49:16 Rebalance progress: 25
2021/12/23 14:49:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:49:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:49:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:49:21 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/23 14:49:21 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:49:21 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 14:49:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 14:49:33 Rebalance progress: 16.66666666666667
2021/12/23 14:49:38 Rebalance progress: 16.66666666666667
2021/12/23 14:49:43 Rebalance progress: 16.66666666666667
2021/12/23 14:49:48 Rebalance progress: 91.96428571428571
2021/12/23 14:49:53 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:49:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:49:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:49:53 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:49:53 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 14:49:58 Rebalance progress: 16.66666666666667
2021/12/23 14:50:03 Rebalance progress: 16.66666666666667
2021/12/23 14:50:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:50:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:50:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:50:08 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:50:08 Waiting for src bucket item count to get to: 200000 curr count: 81852
2021/12/23 14:50:13 Waiting for src bucket item count to get to: 200000 curr count: 98005
2021/12/23 14:50:18 Waiting for src bucket item count to get to: 200000 curr count: 98005
2021/12/23 14:50:23 Waiting for src bucket item count to get to: 200000 curr count: 114410
2021/12/23 14:50:28 Waiting for src bucket item count to get to: 200000 curr count: 114410
2021/12/23 14:50:33 Waiting for src bucket item count to get to: 200000 curr count: 131032
2021/12/23 14:50:38 Waiting for src bucket item count to get to: 200000 curr count: 131032
2021/12/23 14:50:43 Waiting for src bucket item count to get to: 200000 curr count: 147533
2021/12/23 14:50:48 Waiting for src bucket item count to get to: 200000 curr count: 147533
2021/12/23 14:50:53 Waiting for src bucket item count to get to: 200000 curr count: 163844
2021/12/23 14:50:58 Waiting for src bucket item count to get to: 200000 curr count: 163844
2021/12/23 14:51:03 Waiting for src bucket item count to get to: 200000 curr count: 179593
2021/12/23 14:51:08 Waiting for src bucket item count to get to: 200000 curr count: 179593
2021/12/23 14:51:13 Waiting for src bucket item count to get to: 200000 curr count: 196440
2021/12/23 14:51:18 Waiting for src bucket item count to get to: 200000 curr count: 196440
2021/12/23 14:51:23 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 14:51:23 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 14:51:28 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/23 14:51:33 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/23 14:51:38 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/23 14:51:43 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/23 14:51:43 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/23 14:51:43 No eventing-con process running
2021/12/23 14:51:43 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb deleting in background"
}
2021/12/23 14:51:45 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:51:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:51:55 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:51:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:51:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:52:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:52:07 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:52:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingSwapReb (231.00s)
=== RUN   TestDuplicateWithKVRebIn
2021/12/23 14:52:17 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/23 14:52:17 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/23 14:52:17 No eventing-con process running
2021/12/23 14:52:17 Function: TestDuplicateWithKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebIn not found"
}
2021/12/23 14:52:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:52:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:52:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:52:25 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:52:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:52:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:52:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:52:35 Deploying app: TestDuplicateWithKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:52:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn stored in temp store"
}
2021/12/23 14:52:41 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/23 14:52:46 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/23 14:52:51 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/23 14:52:56 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/23 14:52:56 App: TestDuplicateWithKVRebIn got deployed
2021/12/23 14:52:56 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 14:52:56 Starting bucket ops to source bucket
2021/12/23 14:52:57 Removing node(s): [] from the cluster
2021/12/23 14:53:02 Rebalance progress: 16.66666666666667
2021/12/23 14:53:07 Rebalance progress: 16.66666666666667
2021/12/23 14:53:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:53:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:53:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:53:12 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:53:12 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 14:53:17 Rebalance progress: 16.66666666666667
2021/12/23 14:53:22 Rebalance progress: 16.66666666666667
2021/12/23 14:53:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:53:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:53:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:53:27 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:53:27 Waiting for src bucket item count to get to: 200000 curr count: 36291
2021/12/23 14:53:32 Waiting for src bucket item count to get to: 200000 curr count: 52589
2021/12/23 14:53:37 Waiting for src bucket item count to get to: 200000 curr count: 52589
2021/12/23 14:53:42 Waiting for src bucket item count to get to: 200000 curr count: 69320
2021/12/23 14:53:47 Waiting for src bucket item count to get to: 200000 curr count: 69320
2021/12/23 14:53:52 Waiting for src bucket item count to get to: 200000 curr count: 84842
2021/12/23 14:53:57 Waiting for src bucket item count to get to: 200000 curr count: 84842
2021/12/23 14:54:02 Waiting for src bucket item count to get to: 200000 curr count: 101060
2021/12/23 14:54:07 Waiting for src bucket item count to get to: 200000 curr count: 101060
2021/12/23 14:54:12 Waiting for src bucket item count to get to: 200000 curr count: 117317
2021/12/23 14:54:18 Waiting for src bucket item count to get to: 200000 curr count: 117317
2021/12/23 14:54:23 Waiting for src bucket item count to get to: 200000 curr count: 133657
2021/12/23 14:54:28 Waiting for src bucket item count to get to: 200000 curr count: 133657
2021/12/23 14:54:33 Waiting for src bucket item count to get to: 200000 curr count: 149843
2021/12/23 14:54:38 Waiting for src bucket item count to get to: 200000 curr count: 149843
2021/12/23 14:54:43 Waiting for src bucket item count to get to: 200000 curr count: 166009
2021/12/23 14:54:48 Waiting for src bucket item count to get to: 200000 curr count: 166009
2021/12/23 14:54:53 Waiting for src bucket item count to get to: 200000 curr count: 181251
2021/12/23 14:54:58 Waiting for src bucket item count to get to: 200000 curr count: 181251
2021/12/23 14:55:03 Waiting for src bucket item count to get to: 200000 curr count: 181251
2021/12/23 14:55:08 Waiting for src bucket item count to get to: 200000 curr count: 197547
2021/12/23 14:55:13 Waiting for src bucket item count to get to: 200000 curr count: 197547
2021/12/23 14:55:18 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 14:55:18 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 14:55:23 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/23 14:55:28 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/23 14:55:33 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/23 14:55:38 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/23 14:55:38 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/23 14:55:38 No eventing-con process running
2021/12/23 14:55:38 Function: TestDuplicateWithKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn deleting in background"
}
2021/12/23 14:55:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:55:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:55:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:55:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:55:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:55:56 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:55:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebIn (229.67s)
=== RUN   TestDuplicateWithKVRebOut
2021/12/23 14:56:01 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 14:56:02 Removing node(s): [] from the cluster
2021/12/23 14:56:07 Rebalance progress: 100
2021/12/23 14:56:12 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/23 14:56:12 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/23 14:56:12 No eventing-con process running
2021/12/23 14:56:12 Function: TestDuplicateWithKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebOut not found"
}
2021/12/23 14:56:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:56:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:56:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:56:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:56:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:56:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:56:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:56:30 Deploying app: TestDuplicateWithKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 14:56:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut stored in temp store"
}
2021/12/23 14:56:36 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/23 14:56:41 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/23 14:56:46 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/23 14:56:51 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/23 14:56:51 App: TestDuplicateWithKVRebOut got deployed
2021/12/23 14:56:51 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 14:56:51 Starting bucket ops to source bucket
2021/12/23 14:56:56 Rebalance progress: 16.66666666666667
2021/12/23 14:57:01 Rebalance progress: 16.66666666666667
2021/12/23 14:57:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:57:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:57:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 14:57:06 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 14:57:06 Waiting for src bucket item count to get to: 200000 curr count: 12172
2021/12/23 14:57:12 Waiting for src bucket item count to get to: 200000 curr count: 28489
2021/12/23 14:57:17 Waiting for src bucket item count to get to: 200000 curr count: 28489
2021/12/23 14:57:22 Waiting for src bucket item count to get to: 200000 curr count: 44849
2021/12/23 14:57:27 Waiting for src bucket item count to get to: 200000 curr count: 44849
2021/12/23 14:57:32 Waiting for src bucket item count to get to: 200000 curr count: 61656
2021/12/23 14:57:37 Waiting for src bucket item count to get to: 200000 curr count: 61656
2021/12/23 14:57:42 Waiting for src bucket item count to get to: 200000 curr count: 61656
2021/12/23 14:57:47 Waiting for src bucket item count to get to: 200000 curr count: 77871
2021/12/23 14:57:52 Waiting for src bucket item count to get to: 200000 curr count: 77871
2021/12/23 14:57:57 Waiting for src bucket item count to get to: 200000 curr count: 94206
2021/12/23 14:58:02 Waiting for src bucket item count to get to: 200000 curr count: 94206
2021/12/23 14:58:07 Waiting for src bucket item count to get to: 200000 curr count: 110964
2021/12/23 14:58:12 Waiting for src bucket item count to get to: 200000 curr count: 110964
2021/12/23 14:58:17 Waiting for src bucket item count to get to: 200000 curr count: 127543
2021/12/23 14:58:22 Waiting for src bucket item count to get to: 200000 curr count: 127543
2021/12/23 14:58:27 Waiting for src bucket item count to get to: 200000 curr count: 144305
2021/12/23 14:58:32 Waiting for src bucket item count to get to: 200000 curr count: 144305
2021/12/23 14:58:37 Waiting for src bucket item count to get to: 200000 curr count: 160527
2021/12/23 14:58:42 Waiting for src bucket item count to get to: 200000 curr count: 160527
2021/12/23 14:58:47 Waiting for src bucket item count to get to: 200000 curr count: 176086
2021/12/23 14:58:52 Waiting for src bucket item count to get to: 200000 curr count: 176086
2021/12/23 14:58:57 Waiting for src bucket item count to get to: 200000 curr count: 192097
2021/12/23 14:59:02 Waiting for src bucket item count to get to: 200000 curr count: 192097
2021/12/23 14:59:07 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 14:59:07 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 14:59:12 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/23 14:59:17 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/23 14:59:22 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/23 14:59:27 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/23 14:59:27 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/23 14:59:27 No eventing-con process running
2021/12/23 14:59:27 Function: TestDuplicateWithKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut deleting in background"
}
2021/12/23 14:59:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:59:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:59:39 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 14:59:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 14:59:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 14:59:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 14:59:50 Waiting for dst bucket item count to get to: 0 curr count: 92391
2021/12/23 14:59:55 Waiting for dst bucket item count to get to: 0 curr count: 92391
2021/12/23 14:59:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebOut (239.05s)
=== RUN   TestDuplicateWithKVSwapReb
2021/12/23 15:00:05 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/23 15:00:05 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/23 15:00:05 No eventing-con process running
2021/12/23 15:00:05 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVSwapReb not found"
}
2021/12/23 15:00:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:00:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:00:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:00:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:00:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:00:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:00:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:00:24 Deploying app: TestDuplicateWithKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:00:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb stored in temp store"
}
2021/12/23 15:00:30 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/23 15:00:35 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/23 15:00:40 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/23 15:00:45 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/23 15:00:45 App: TestDuplicateWithKVSwapReb got deployed
2021/12/23 15:00:45 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 15:00:45 Starting bucket ops to source bucket
2021/12/23 15:00:45 Removing node(s): [] from the cluster
2021/12/23 15:00:50 Rebalance progress: 16.66666666666667
2021/12/23 15:00:55 Rebalance progress: 16.66666666666667
2021/12/23 15:01:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:01:00 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:01:00 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 15:01:01 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:01:06 Rebalance progress: 16.66666666666667
2021/12/23 15:01:11 Rebalance progress: 16.66666666666667
2021/12/23 15:01:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:01:16 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:01:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:01:21 Rebalance progress: 16.66666666666667
2021/12/23 15:01:26 Rebalance progress: 16.66666666666667
2021/12/23 15:01:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:01:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:01:31 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:01:31 Waiting for src bucket item count to get to: 200000 curr count: 70022
2021/12/23 15:01:36 Waiting for src bucket item count to get to: 200000 curr count: 70022
2021/12/23 15:01:41 Waiting for src bucket item count to get to: 200000 curr count: 85125
2021/12/23 15:01:46 Waiting for src bucket item count to get to: 200000 curr count: 85125
2021/12/23 15:01:51 Waiting for src bucket item count to get to: 200000 curr count: 101625
2021/12/23 15:01:56 Waiting for src bucket item count to get to: 200000 curr count: 101625
2021/12/23 15:02:01 Waiting for src bucket item count to get to: 200000 curr count: 118303
2021/12/23 15:02:06 Waiting for src bucket item count to get to: 200000 curr count: 118303
2021/12/23 15:02:11 Waiting for src bucket item count to get to: 200000 curr count: 134738
2021/12/23 15:02:16 Waiting for src bucket item count to get to: 200000 curr count: 134738
2021/12/23 15:02:21 Waiting for src bucket item count to get to: 200000 curr count: 151173
2021/12/23 15:02:26 Waiting for src bucket item count to get to: 200000 curr count: 151173
2021/12/23 15:02:31 Waiting for src bucket item count to get to: 200000 curr count: 151173
2021/12/23 15:02:36 Waiting for src bucket item count to get to: 200000 curr count: 167747
2021/12/23 15:02:41 Waiting for src bucket item count to get to: 200000 curr count: 167747
2021/12/23 15:02:46 Waiting for src bucket item count to get to: 200000 curr count: 183337
2021/12/23 15:02:51 Waiting for src bucket item count to get to: 200000 curr count: 183337
2021/12/23 15:02:56 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 15:02:56 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 15:03:01 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/23 15:03:06 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/23 15:03:11 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/23 15:03:16 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/23 15:03:16 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/23 15:03:16 No eventing-con process running
2021/12/23 15:03:16 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb deleting in background"
}
2021/12/23 15:03:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:03:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:03:28 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/23 15:03:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:03:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:03:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:03:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVSwapReb (219.43s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpOneByOne
2021/12/23 15:03:45 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:03:45 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:03:45 No eventing-con process running
2021/12/23 15:03:45 Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/23 15:03:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:03:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:03:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:03:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:03:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:03:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:04:03 Deploying app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:04:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/23 15:04:09 Starting bucket ops to source bucket
2021/12/23 15:04:14 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:04:19 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:04:24 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:04:24 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:04:24 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:04:24 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:04:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:04:30 Rebalance progress: 16.66666666666667
2021/12/23 15:04:35 Rebalance progress: 16.66666666666667
2021/12/23 15:04:40 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:04:40 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:04:40 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:04:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:04:45 Rebalance progress: 16.66666666666667
2021/12/23 15:04:50 Rebalance progress: 16.66666666666667
2021/12/23 15:04:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:04:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:04:55 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/23 15:04:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 15:05:01 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:05:06 Rebalance progress: 12.5
2021/12/23 15:05:11 Rebalance progress: 12.5
2021/12/23 15:05:16 Rebalance progress: 12.5
2021/12/23 15:05:21 Rebalance progress: 72
2021/12/23 15:05:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:05:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:05:31 Rebalance progress: 12.5
2021/12/23 15:05:36 Rebalance progress: 12.5
2021/12/23 15:05:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:05:41 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:05:46 Rebalance progress: 12.5
2021/12/23 15:05:51 Rebalance progress: 12.5
2021/12/23 15:05:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:05:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:05:56 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 15:06:01 Rebalance progress: 12.5
2021/12/23 15:06:06 Rebalance progress: 12.5
2021/12/23 15:06:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:06:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:06:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:06:16 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:06:21 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:06:26 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:06:31 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:06:31 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:06:31 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:06:31 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/23 15:06:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:06:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:06:43 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12172
2021/12/23 15:06:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12172
2021/12/23 15:06:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:06:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:06:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:07:00 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 505
2021/12/23 15:07:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 505
2021/12/23 15:07:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpOneByOne (210.14s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce
2021/12/23 15:07:15 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/23 15:07:15 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/23 15:07:15 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:07:15 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce not found"
}
2021/12/23 15:07:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:07:22 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:07:22 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:07:23 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:07:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:07:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:07:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:07:33 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:07:35 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce stored in temp store"
}
2021/12/23 15:07:40 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:07:45 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/23 15:07:50 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/23 15:07:55 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/23 15:07:55 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:07:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:07:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:07:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:07:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:07:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 15:07:55 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:08:00 Rebalance progress: 12.5
2021/12/23 15:08:05 Rebalance progress: 12.5
2021/12/23 15:08:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:08:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:08:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:08:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/23 15:08:15 Rebalance progress: 12.5
2021/12/23 15:08:20 Rebalance progress: 12.5
2021/12/23 15:08:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:08:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:08:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:08:31 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/23 15:08:36 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/23 15:08:41 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/23 15:08:46 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/23 15:08:46 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/23 15:08:46 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:08:46 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce deleting in background"
}
2021/12/23 15:08:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:08:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:08:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4571
2021/12/23 15:08:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:08:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:09:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4571
2021/12/23 15:09:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce (124.30s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings
2021/12/23 15:09:19 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 15:09:19 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/23 15:09:19 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:09:19 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/23 15:09:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:09:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:09:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:09:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:09:38 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:09:39 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/23 15:09:44 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:09:49 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/23 15:09:54 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/23 15:09:59 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/23 15:09:59 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:09:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:09:59 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:09:59 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:10:04 Rebalance progress: 12.5
2021/12/23 15:10:09 Rebalance progress: 12.5
2021/12/23 15:10:14 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:10:14 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:10:15 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:10:20 Rebalance progress: 12.5
2021/12/23 15:10:25 Rebalance progress: 12.5
2021/12/23 15:10:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:10:30 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 15:10:30 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:10:35 Rebalance progress: 12.5
2021/12/23 15:10:40 Rebalance progress: 12.5
2021/12/23 15:10:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:10:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:10:46 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:10:51 Rebalance progress: 12.5
2021/12/23 15:10:56 Rebalance progress: 12.5
2021/12/23 15:11:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:11:01 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:11:06 Rebalance progress: 12.5
2021/12/23 15:11:11 Rebalance progress: 12.5
2021/12/23 15:11:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:11:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 15:11:21 Rebalance progress: 12.5
2021/12/23 15:11:26 Rebalance progress: 12.5
2021/12/23 15:11:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:11:36 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 15:11:41 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 15:11:46 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 15:11:51 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/23 15:11:51 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/23 15:11:51 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:11:51 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/23 15:11:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:11:58 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:12:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10687
2021/12/23 15:12:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10687
2021/12/23 15:12:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:12:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:12:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings (185.34s)
=== RUN   TestEventingSwapRebOnUpdateBucketOp
2021/12/23 15:12:25 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/23 15:12:25 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/23 15:12:25 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:12:25 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebOnUpdateBucketOp not found"
}
2021/12/23 15:12:26 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:31 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:12:31 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:12:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:12:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:12:38 eventingSetup.go:125: Deploying app: TestEventingSwapRebOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:12:39 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp stored in temp store"
}
2021/12/23 15:12:44 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:12:49 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/23 15:12:54 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/23 15:12:59 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/23 15:12:59 clusterSetup.go:354: App: TestEventingSwapRebOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:12:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:12:59 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:13:00 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:13:05 Rebalance progress: 12.5
2021/12/23 15:13:10 Rebalance progress: 12.5
2021/12/23 15:13:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:13:15 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:13:15 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:13:20 Rebalance progress: 12.5
2021/12/23 15:13:25 Rebalance progress: 12.5
2021/12/23 15:13:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:13:30 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:13:35 Rebalance progress: 12.5
2021/12/23 15:13:40 Rebalance progress: 12.5
2021/12/23 15:13:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:13:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:13:50 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/23 15:13:56 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/23 15:14:01 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/23 15:14:01 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/23 15:14:01 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:14:01 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp deleting in background"
}
2021/12/23 15:14:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:14:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:14:12 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6107
2021/12/23 15:14:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6107
2021/12/23 15:14:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:14:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:14:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:14:29 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6107
2021/12/23 15:14:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebOnUpdateBucketOp (134.58s)
=== RUN   TestMetaRollbackWithEventingReb
2021/12/23 15:14:34 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:14:34 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:14:40 Rebalance progress: 100
2021/12/23 15:14:45 Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/23 15:14:45 App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/23 15:14:45 No eventing-con process running
2021/12/23 15:14:45 Function: TestMetaRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaRollbackWithEventingReb not found"
}
2021/12/23 15:14:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:14:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:14:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:14:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:14:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:14:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:14:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:14:58 Deploying app: TestMetaRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:14:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb stored in temp store"
}
2021/12/23 15:15:04 Starting bucket ops to source bucket
2021/12/23 15:15:09 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/23 15:15:14 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/23 15:15:19 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/23 15:15:19 App: TestMetaRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:15:19 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:15:20 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:15:25 Rebalance progress: 12.5
2021/12/23 15:15:30 Rebalance progress: 12.5
2021/12/23 15:15:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:15:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/23 15:15:35 bucketOps.go:134: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/23 15:15:40 Rebalance progress: 12.5
2021/12/23 15:15:45 Rebalance progress: 12.5
2021/12/23 15:15:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:15:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:15:55 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/23 15:16:00 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/23 15:16:05 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/23 15:16:10 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/23 15:16:10 clusterSetup.go:405: App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/23 15:16:10 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:16:10 eventingSetup.go:418: Function: TestMetaRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb deleting in background"
}
2021/12/23 15:16:12 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:16:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:16:22 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4552
2021/12/23 15:16:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4552
2021/12/23 15:16:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:16:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:16:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:16:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4552
2021/12/23 15:16:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaRollbackWithEventingReb (129.39s)
=== RUN   TestMetaPartialRollbackWithEventingReb
2021/12/23 15:16:44 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:16:44 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:16:49 Rebalance progress: 100
2021/12/23 15:16:54 Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/23 15:16:54 App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/23 15:16:54 No eventing-con process running
2021/12/23 15:16:54 Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaPartialRollbackWithEventingReb not found"
}
2021/12/23 15:16:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:17:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:17:02 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:17:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:17:07 Deploying app: TestMetaPartialRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:17:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb stored in temp store"
}
2021/12/23 15:17:13 Starting bucket ops to source bucket
2021/12/23 15:17:18 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/23 15:17:23 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/23 15:17:29 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/23 15:17:29 App: TestMetaPartialRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:17:29 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:17:29 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:17:34 Rebalance progress: 12.5
2021/12/23 15:17:39 Rebalance progress: 12.5
2021/12/23 15:17:44 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:17:44 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/23 15:17:44 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/23 15:17:49 Rebalance progress: 12.5
2021/12/23 15:17:54 Rebalance progress: 12.5
2021/12/23 15:17:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:17:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:17:59 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/23 15:18:04 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/23 15:18:09 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/23 15:18:14 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/23 15:18:19 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/23 15:18:19 clusterSetup.go:405: App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/23 15:18:19 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:18:19 eventingSetup.go:418: Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb deleting in background"
}
2021/12/23 15:18:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:18:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:18:31 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 38371
2021/12/23 15:18:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 38371
2021/12/23 15:18:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:18:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:18:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:18:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 38371
2021/12/23 15:18:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaPartialRollbackWithEventingReb (129.27s)
=== RUN   TestEventingRebMultipleHandlersOneByOne
2021/12/23 15:18:58 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/23 15:18:58 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/23 15:18:58 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:18:58 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op not found"
}
2021/12/23 15:19:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:19:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:19:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:19:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:19:16 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:19:17 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:19:17 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:19:19 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update stored in temp store"
}
2021/12/23 15:19:24 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:19:29 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/23 15:19:34 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/23 15:19:39 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/23 15:19:39 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got deployed
2021/12/23 15:19:44 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/23 15:19:49 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/23 15:19:54 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/23 15:19:54 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:19:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:19:54 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:19:54 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:19:59 Rebalance progress: 12.5
2021/12/23 15:20:04 Rebalance progress: 12.5
2021/12/23 15:20:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:20:09 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:20:09 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:20:14 Rebalance progress: 12.5
2021/12/23 15:20:19 Rebalance progress: 12.5
2021/12/23 15:20:24 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:20:24 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 15:20:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:20:30 Rebalance progress: 12.5
2021/12/23 15:20:35 Rebalance progress: 12.5
2021/12/23 15:20:40 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:20:40 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:20:45 Rebalance progress: 12.5
2021/12/23 15:20:50 Rebalance progress: 12.5
2021/12/23 15:20:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:20:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:20:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:21:00 Rebalance progress: 12.5
2021/12/23 15:21:05 Rebalance progress: 12.5
2021/12/23 15:21:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:21:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:21:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:21:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 15:21:15 Rebalance progress: 12.5
2021/12/23 15:21:20 Rebalance progress: 12.5
2021/12/23 15:21:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:21:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:21:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:21:30 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/23 15:21:36 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/23 15:21:41 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/23 15:21:46 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/23 15:21:46 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/23 15:21:46 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:21:46 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op deleting in background"
}
2021/12/23 15:21:49 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:21:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:21:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12167
2021/12/23 15:21:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:22:01 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:22:06 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:22:11 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7648
2021/12/23 15:22:16 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7648
2021/12/23 15:22:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:22:21 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/23 15:22:26 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/23 15:22:31 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/23 15:22:36 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/23 15:22:36 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got un-deployed
2021/12/23 15:22:36 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:22:36 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update deleting in background"
}
2021/12/23 15:22:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:22:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:22:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:22:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:22:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:22:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersOneByOne (241.56s)
=== RUN   TestEventingRebMultipleHandlersAllAtOnce
2021/12/23 15:22:59 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/23 15:22:59 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/23 15:22:59 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:22:59 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op not found"
}
2021/12/23 15:23:01 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:06 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:23:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:23:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:23:13 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:23:18 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:23:19 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:19 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:23:19 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:23:20 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update stored in temp store"
}
2021/12/23 15:23:25 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:23:30 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/23 15:23:35 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/23 15:23:40 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/23 15:23:40 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got deployed
2021/12/23 15:23:45 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/23 15:23:50 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/23 15:23:55 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/23 15:23:55 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:23:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:23:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:23:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:23:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/23 15:23:56 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:24:01 Rebalance progress: 12.5
2021/12/23 15:24:06 Rebalance progress: 12.5
2021/12/23 15:24:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:24:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:24:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:24:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/23 15:24:16 Rebalance progress: 12.5
2021/12/23 15:24:21 Rebalance progress: 12.5
2021/12/23 15:24:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:24:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:24:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:24:32 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/23 15:24:37 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/23 15:24:42 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/23 15:24:47 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/23 15:24:47 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/23 15:24:47 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:24:47 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op deleting in background"
}
2021/12/23 15:24:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:24:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:24:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:24:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:25:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:25:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6119
2021/12/23 15:25:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:25:12 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/23 15:25:17 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/23 15:25:22 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/23 15:25:27 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/23 15:25:27 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got un-deployed
2021/12/23 15:25:27 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:25:27 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update deleting in background"
}
2021/12/23 15:25:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:25:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:25:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:25:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:25:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:25:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersAllAtOnce (171.28s)
=== RUN   TestEventingSwapRebMultipleHandlers
2021/12/23 15:25:51 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/23 15:25:51 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/23 15:25:51 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:25:51 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update not found"
}
2021/12/23 15:25:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:25:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:25:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:25:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:26:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:26:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:26:09 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/23 15:26:09 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/23 15:26:09 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:26:09 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op not found"
}
2021/12/23 15:26:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:26:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:26:16 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:26:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:26:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:26:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:26:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:26:23 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlersbucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:26:24 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:26:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:26:24 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:26:25 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op stored in temp store"
}
2021/12/23 15:26:30 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:26:35 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/23 15:26:40 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/23 15:26:45 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/23 15:26:45 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got deployed
2021/12/23 15:26:50 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/23 15:26:55 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/23 15:27:00 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/23 15:27:00 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:27:00 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:27:01 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:27:06 Rebalance progress: 12.5
2021/12/23 15:27:11 Rebalance progress: 12.5
2021/12/23 15:27:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:27:16 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:27:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:27:21 Rebalance progress: 12.5
2021/12/23 15:27:26 Rebalance progress: 12.5
2021/12/23 15:27:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:27:31 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:27:36 Rebalance progress: 12.5
2021/12/23 15:27:41 Rebalance progress: 12.5
2021/12/23 15:27:46 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:27:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:27:51 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/23 15:27:57 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/23 15:28:02 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/23 15:28:07 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/23 15:28:07 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/23 15:28:07 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:28:07 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update deleting in background"
}
2021/12/23 15:28:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:28:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:28:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:28:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:28:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:28:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7631
2021/12/23 15:28:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:28:32 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/23 15:28:37 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/23 15:28:42 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/23 15:28:47 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/23 15:28:47 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/23 15:28:47 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:28:47 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op deleting in background"
}
2021/12/23 15:28:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:28:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:28:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:28:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:29:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebMultipleHandlers (199.53s)
=== RUN   TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/23 15:29:10 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:29:10 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:29:10 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:29:10 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/23 15:29:12 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:29:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:29:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:29:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:29:29 eventingSetup.go:125: Deploying app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:29:30 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/23 15:29:35 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:29:40 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:29:45 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:29:50 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:29:50 clusterSetup.go:354: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:29:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:29:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:29:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:30:10 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:30:41 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:30:46 Rebalance progress: 12.5
2021/12/23 15:30:51 Rebalance progress: 12.5
2021/12/23 15:30:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:30:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:30:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:30:57 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:31:02 Rebalance progress: 12.5
2021/12/23 15:31:07 Rebalance progress: 12.5
2021/12/23 15:31:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:31:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:31:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:31:12 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:31:17 Rebalance progress: 12.5
2021/12/23 15:31:22 Rebalance progress: 12.5
2021/12/23 15:31:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:31:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:31:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:31:32 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:31:37 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:31:42 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:31:47 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:31:47 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:31:47 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:31:47 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/23 15:31:49 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:31:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:31:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11174
2021/12/23 15:31:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:32:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:32:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:32:10 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5889
2021/12/23 15:32:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5889
2021/12/23 15:32:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne (195.24s)
=== RUN   TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/23 15:32:25 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:32:25 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:32:25 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:32:25 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/23 15:32:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:32:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:32:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:32:34 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:32:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:32:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:32:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:32:44 eventingSetup.go:125: Deploying app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:32:45 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/23 15:32:50 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:32:55 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:33:00 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:33:05 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/23 15:33:05 clusterSetup.go:354: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:33:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:33:05 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:33:25 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:36 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:36 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:33:36 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:33:37 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:33:57 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:33:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:34:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:34:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:34:28 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:34:38 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:34:39 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:34:59 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:34:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:35:09 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:35:09 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:35:30 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:35:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:35:45 Rebalance progress: 12.5
2021/12/23 15:35:50 Rebalance progress: 12.5
2021/12/23 15:35:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:35:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:35:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:35:56 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:36:01 Rebalance progress: 12.5
2021/12/23 15:36:06 Rebalance progress: 12.5
2021/12/23 15:36:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:36:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:36:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:36:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:36:16 Rebalance progress: 12.5
2021/12/23 15:36:21 Rebalance progress: 12.5
2021/12/23 15:36:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:36:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:36:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:36:31 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:36:36 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:36:41 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:36:46 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/23 15:36:46 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/23 15:36:46 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:36:46 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/23 15:36:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:36:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:36:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 21558
2021/12/23 15:36:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:37:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:37:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne (289.21s)
=== RUN   TestEventingFailoverOnUpdateBucketOp
2021/12/23 15:37:15 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/23 15:37:15 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/23 15:37:15 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:37:15 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingFailoverOnUpdateBucketOp not found"
}
2021/12/23 15:37:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:37:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:37:23 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:28 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:37:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:37:33 eventingSetup.go:125: Deploying app: TestEventingFailoverOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:37:34 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp stored in temp store"
}
2021/12/23 15:37:39 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:37:44 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/23 15:37:49 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/23 15:37:54 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/23 15:37:54 clusterSetup.go:354: App: TestEventingFailoverOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:37:54 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:37:54 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:37:55 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:38:00 Rebalance progress: 12.5
2021/12/23 15:38:05 Rebalance progress: 12.5
2021/12/23 15:38:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:38:10 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:38:20 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:38:25 Rebalance progress: 12.5
2021/12/23 15:38:30 Rebalance progress: 12.5
2021/12/23 15:38:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:38:35 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:38:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:38:40 Rebalance progress: 12.5
2021/12/23 15:38:45 Rebalance progress: 12.5
2021/12/23 15:38:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:38:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:38:50 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:38:55 Rebalance progress: 12.5
2021/12/23 15:39:00 Rebalance progress: 12.5
2021/12/23 15:39:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:39:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:39:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:39:11 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/23 15:39:16 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/23 15:39:21 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/23 15:39:26 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/23 15:39:26 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/23 15:39:26 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:39:26 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp deleting in background"
}
2021/12/23 15:39:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:39:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:39:37 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8609
2021/12/23 15:39:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:39:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:39:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:39:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8609
2021/12/23 15:39:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingFailoverOnUpdateBucketOp (164.50s)
=== RUN   TestEventingKVRebalanceOnUpdateBucketOp
2021/12/23 15:39:59 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/23 15:39:59 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/23 15:39:59 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:39:59 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingKVRebalanceOnUpdateBucketOp not found"
}
2021/12/23 15:40:01 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:06 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:40:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:40:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:40:13 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:40:18 eventingSetup.go:125: Deploying app: TestEventingKVRebalanceOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:40:19 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp stored in temp store"
}
2021/12/23 15:40:24 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:40:29 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/23 15:40:34 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/23 15:40:39 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/23 15:40:39 clusterSetup.go:354: App: TestEventingKVRebalanceOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:40:39 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing,kv to the cluster
2021/12/23 15:40:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:40:45 Rebalance progress: 12.5
2021/12/23 15:40:50 Rebalance progress: 12.5
2021/12/23 15:40:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:40:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:40:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing,kv to the cluster
2021/12/23 15:40:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:41:00 Rebalance progress: 12.5
2021/12/23 15:41:05 Rebalance progress: 12.5
2021/12/23 15:41:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:41:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:41:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:41:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:41:15 Rebalance progress: 12.5
2021/12/23 15:41:20 Rebalance progress: 12.5
2021/12/23 15:41:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:41:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:41:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:41:30 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/23 15:41:35 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/23 15:41:40 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/23 15:41:45 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/23 15:41:46 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/23 15:41:46 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:41:46 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp deleting in background"
}
2021/12/23 15:41:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:41:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:41:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6125
2021/12/23 15:41:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:41:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:42:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:42:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6125
2021/12/23 15:42:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingKVRebalanceOnUpdateBucketOp (139.92s)
=== RUN   TestEventingRebInWithOps
2021/12/23 15:42:19 clusterSetup.go:397: Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/23 15:42:19 clusterSetup.go:405: App: TestEventingRebInWithOps got un-deployed
2021/12/23 15:42:19 eventingSetup.go:46: No eventing-con process running
2021/12/23 15:42:19 eventingSetup.go:418: Function: TestEventingRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithOps not found"
}
2021/12/23 15:42:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:42:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:42:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:42:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:42:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:42:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:42:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:42:38 eventingSetup.go:125: Deploying app: TestEventingRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:42:39 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps stored in temp store"
}
2021/12/23 15:42:44 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/23 15:42:49 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/23 15:42:54 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/23 15:42:59 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/23 15:42:59 clusterSetup.go:354: App: TestEventingRebInWithOps got deployed
2021/12/23 15:42:59 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:42:59 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 15:42:59 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 15:43:04 Rebalance progress: 12.5
2021/12/23 15:43:09 Rebalance progress: 12.5
2021/12/23 15:43:14 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:43:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:43:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:43:14 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:43:19 Rebalance progress: 12.5
2021/12/23 15:43:24 Rebalance progress: 12.5
2021/12/23 15:43:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:43:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:43:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:43:29 Waiting for src bucket item count to get to: 200000 curr count: 31678
2021/12/23 15:43:34 Waiting for src bucket item count to get to: 200000 curr count: 47676
2021/12/23 15:43:39 Waiting for src bucket item count to get to: 200000 curr count: 47676
2021/12/23 15:43:44 Waiting for src bucket item count to get to: 200000 curr count: 63665
2021/12/23 15:43:49 Waiting for src bucket item count to get to: 200000 curr count: 63665
2021/12/23 15:43:54 Waiting for src bucket item count to get to: 200000 curr count: 78949
2021/12/23 15:43:59 Waiting for src bucket item count to get to: 200000 curr count: 78949
2021/12/23 15:44:04 Waiting for src bucket item count to get to: 200000 curr count: 95015
2021/12/23 15:44:09 Waiting for src bucket item count to get to: 200000 curr count: 95015
2021/12/23 15:44:14 Waiting for src bucket item count to get to: 200000 curr count: 111339
2021/12/23 15:44:19 Waiting for src bucket item count to get to: 200000 curr count: 111339
2021/12/23 15:44:24 Waiting for src bucket item count to get to: 200000 curr count: 127725
2021/12/23 15:44:29 Waiting for src bucket item count to get to: 200000 curr count: 127725
2021/12/23 15:44:35 Waiting for src bucket item count to get to: 200000 curr count: 127725
2021/12/23 15:44:40 Waiting for src bucket item count to get to: 200000 curr count: 144239
2021/12/23 15:44:45 Waiting for src bucket item count to get to: 200000 curr count: 160611
2021/12/23 15:44:50 Waiting for src bucket item count to get to: 200000 curr count: 160611
2021/12/23 15:44:55 Waiting for src bucket item count to get to: 200000 curr count: 175847
2021/12/23 15:45:00 Waiting for src bucket item count to get to: 200000 curr count: 175847
2021/12/23 15:45:05 Waiting for src bucket item count to get to: 200000 curr count: 192035
2021/12/23 15:45:10 Waiting for src bucket item count to get to: 200000 curr count: 192035
2021/12/23 15:45:15 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 15:45:15 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 15:45:15 Starting bucket ops to source bucket
2021/12/23 15:49:00 Waiting for dst bucket item count to get to: 0 curr count: 10362
2021/12/23 15:49:05 Waiting for dst bucket item count to get to: 0 curr count: 10362
2021/12/23 15:49:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:49:05 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 15:49:10 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/23 15:49:15 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/23 15:49:20 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/23 15:49:25 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/23 15:49:25 App: TestEventingRebInWithOps got un-deployed
2021/12/23 15:49:25 No eventing-con process running
2021/12/23 15:49:25 Function: TestEventingRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps deleting in background"
}
2021/12/23 15:49:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:49:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:49:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:49:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:49:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:49:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithOps (449.67s)
=== RUN   TestEventingRebOutWithOps
2021/12/23 15:49:44 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:49:44 Removing node(s): [] from the cluster
2021/12/23 15:49:49 Rebalance progress: 100
2021/12/23 15:49:54 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/23 15:49:54 App: TestEventingRebOutWithOps got un-deployed
2021/12/23 15:49:54 No eventing-con process running
2021/12/23 15:49:54 Function: TestEventingRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithOps not found"
}
2021/12/23 15:49:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:50:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:50:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:50:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:50:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:50:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:50:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:50:13 Deploying app: TestEventingRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:50:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps stored in temp store"
}
2021/12/23 15:50:19 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/23 15:50:24 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/23 15:50:29 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/23 15:50:34 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/23 15:50:34 App: TestEventingRebOutWithOps got deployed
2021/12/23 15:50:34 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:50:34 Starting bucket ops to source bucket
2021/12/23 15:50:39 Rebalance progress: 12.5
2021/12/23 15:50:44 Rebalance progress: 12.5
2021/12/23 15:50:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:50:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:50:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:50:49 Waiting for src bucket item count to get to: 200000 curr count: 7638
2021/12/23 15:50:54 Waiting for src bucket item count to get to: 200000 curr count: 23560
2021/12/23 15:50:59 Waiting for src bucket item count to get to: 200000 curr count: 23560
2021/12/23 15:51:04 Waiting for src bucket item count to get to: 200000 curr count: 40066
2021/12/23 15:51:09 Waiting for src bucket item count to get to: 200000 curr count: 40066
2021/12/23 15:51:14 Waiting for src bucket item count to get to: 200000 curr count: 56525
2021/12/23 15:51:19 Waiting for src bucket item count to get to: 200000 curr count: 56525
2021/12/23 15:51:24 Waiting for src bucket item count to get to: 200000 curr count: 56525
2021/12/23 15:51:29 Waiting for src bucket item count to get to: 200000 curr count: 73019
2021/12/23 15:51:34 Waiting for src bucket item count to get to: 200000 curr count: 73019
2021/12/23 15:51:39 Waiting for src bucket item count to get to: 200000 curr count: 88224
2021/12/23 15:51:44 Waiting for src bucket item count to get to: 200000 curr count: 88224
2021/12/23 15:51:49 Waiting for src bucket item count to get to: 200000 curr count: 104563
2021/12/23 15:51:54 Waiting for src bucket item count to get to: 200000 curr count: 120729
2021/12/23 15:51:59 Waiting for src bucket item count to get to: 200000 curr count: 120729
2021/12/23 15:52:04 Waiting for src bucket item count to get to: 200000 curr count: 137210
2021/12/23 15:52:09 Waiting for src bucket item count to get to: 200000 curr count: 137210
2021/12/23 15:52:14 Waiting for src bucket item count to get to: 200000 curr count: 153651
2021/12/23 15:52:19 Waiting for src bucket item count to get to: 200000 curr count: 153651
2021/12/23 15:52:24 Waiting for src bucket item count to get to: 200000 curr count: 170174
2021/12/23 15:52:29 Waiting for src bucket item count to get to: 200000 curr count: 170174
2021/12/23 15:52:34 Waiting for src bucket item count to get to: 200000 curr count: 185402
2021/12/23 15:52:39 Waiting for src bucket item count to get to: 200000 curr count: 185402
2021/12/23 15:52:44 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 15:52:44 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 15:52:44 Starting bucket ops to source bucket
2021/12/23 15:56:29 Waiting for dst bucket item count to get to: 0 curr count: 8722
2021/12/23 15:56:34 Waiting for dst bucket item count to get to: 0 curr count: 8722
2021/12/23 15:56:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:56:34 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 15:56:39 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/23 15:56:45 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/23 15:56:50 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/23 15:56:55 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/23 15:56:55 App: TestEventingRebOutWithOps got un-deployed
2021/12/23 15:56:55 No eventing-con process running
2021/12/23 15:56:55 Function: TestEventingRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps deleting in background"
}
2021/12/23 15:56:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:57:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:57:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:57:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:57:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:57:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithOps (449.46s)
=== RUN   TestEventingSwapRebWithOps
2021/12/23 15:57:18 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/23 15:57:18 App: TestEventingSwapRebWithOps got un-deployed
2021/12/23 15:57:18 No eventing-con process running
2021/12/23 15:57:18 Function: TestEventingSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithOps not found"
}
2021/12/23 15:57:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:57:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:57:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 15:57:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:57:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:57:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:57:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 15:57:37 Deploying app: TestEventingSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 15:57:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps stored in temp store"
}
2021/12/23 15:57:43 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/23 15:57:48 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/23 15:57:53 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/23 15:57:58 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/23 15:57:58 App: TestEventingSwapRebWithOps got deployed
2021/12/23 15:57:58 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 15:57:58 Starting bucket ops to source bucket
2021/12/23 15:57:58 Removing node(s): [] from the cluster
2021/12/23 15:58:03 Rebalance progress: 12.5
2021/12/23 15:58:08 Rebalance progress: 12.5
2021/12/23 15:58:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:58:13 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 15:58:14 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 15:58:19 Rebalance progress: 12.5
2021/12/23 15:58:24 Rebalance progress: 12.5
2021/12/23 15:58:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:58:29 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 15:58:34 Rebalance progress: 12.5
2021/12/23 15:58:39 Rebalance progress: 12.5
2021/12/23 15:58:44 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 15:58:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 15:58:44 Waiting for src bucket item count to get to: 200000 curr count: 64867
2021/12/23 15:58:49 Waiting for src bucket item count to get to: 200000 curr count: 64867
2021/12/23 15:58:54 Waiting for src bucket item count to get to: 200000 curr count: 79919
2021/12/23 15:58:59 Waiting for src bucket item count to get to: 200000 curr count: 79919
2021/12/23 15:59:04 Waiting for src bucket item count to get to: 200000 curr count: 96067
2021/12/23 15:59:09 Waiting for src bucket item count to get to: 200000 curr count: 96067
2021/12/23 15:59:14 Waiting for src bucket item count to get to: 200000 curr count: 112413
2021/12/23 15:59:19 Waiting for src bucket item count to get to: 200000 curr count: 112413
2021/12/23 15:59:24 Waiting for src bucket item count to get to: 200000 curr count: 128901
2021/12/23 15:59:29 Waiting for src bucket item count to get to: 200000 curr count: 128901
2021/12/23 15:59:34 Waiting for src bucket item count to get to: 200000 curr count: 145175
2021/12/23 15:59:39 Waiting for src bucket item count to get to: 200000 curr count: 145175
2021/12/23 15:59:44 Waiting for src bucket item count to get to: 200000 curr count: 145175
2021/12/23 15:59:49 Waiting for src bucket item count to get to: 200000 curr count: 161635
2021/12/23 15:59:54 Waiting for src bucket item count to get to: 200000 curr count: 161635
2021/12/23 15:59:59 Waiting for src bucket item count to get to: 200000 curr count: 177185
2021/12/23 16:00:04 Waiting for src bucket item count to get to: 200000 curr count: 177185
2021/12/23 16:00:09 Waiting for src bucket item count to get to: 200000 curr count: 193415
2021/12/23 16:00:14 Waiting for src bucket item count to get to: 200000 curr count: 193415
2021/12/23 16:00:19 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 16:00:19 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 16:00:19 Starting bucket ops to source bucket
2021/12/23 16:04:05 Waiting for dst bucket item count to get to: 0 curr count: 19660
2021/12/23 16:04:10 Waiting for dst bucket item count to get to: 0 curr count: 1095
2021/12/23 16:04:15 Waiting for dst bucket item count to get to: 0 curr count: 1095
2021/12/23 16:04:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:04:15 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:04:20 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/23 16:04:25 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/23 16:04:30 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/23 16:04:35 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/23 16:04:35 App: TestEventingSwapRebWithOps got un-deployed
2021/12/23 16:04:35 No eventing-con process running
2021/12/23 16:04:35 Function: TestEventingSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps deleting in background"
}
2021/12/23 16:04:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:04:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:04:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:04:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:04:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:04:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithOps (460.47s)
=== RUN   TestKVRebInWithOps
2021/12/23 16:04:59 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/23 16:04:59 App: TestKVRebInWithOps got un-deployed
2021/12/23 16:04:59 No eventing-con process running
2021/12/23 16:04:59 Function: TestKVRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithOps not found"
}
2021/12/23 16:05:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:05:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:05:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:05:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:05:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:05:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:05:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:05:17 Deploying app: TestKVRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:05:18 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps stored in temp store"
}
2021/12/23 16:05:23 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/23 16:05:28 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/23 16:05:33 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/23 16:05:38 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/23 16:05:38 App: TestKVRebInWithOps got deployed
2021/12/23 16:05:38 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:05:38 Starting bucket ops to source bucket
2021/12/23 16:05:39 Removing node(s): [] from the cluster
2021/12/23 16:05:44 Rebalance progress: 12.5
2021/12/23 16:05:49 Rebalance progress: 12.5
2021/12/23 16:05:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:05:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:05:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:05:54 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:05:59 Rebalance progress: 12.5
2021/12/23 16:06:04 Rebalance progress: 12.5
2021/12/23 16:06:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:06:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:06:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:06:09 Waiting for src bucket item count to get to: 200000 curr count: 31943
2021/12/23 16:06:14 Waiting for src bucket item count to get to: 200000 curr count: 48189
2021/12/23 16:06:19 Waiting for src bucket item count to get to: 200000 curr count: 48189
2021/12/23 16:06:24 Waiting for src bucket item count to get to: 200000 curr count: 64825
2021/12/23 16:06:29 Waiting for src bucket item count to get to: 200000 curr count: 64825
2021/12/23 16:06:34 Waiting for src bucket item count to get to: 200000 curr count: 80327
2021/12/23 16:06:39 Waiting for src bucket item count to get to: 200000 curr count: 80327
2021/12/23 16:06:44 Waiting for src bucket item count to get to: 200000 curr count: 96597
2021/12/23 16:06:49 Waiting for src bucket item count to get to: 200000 curr count: 96597
2021/12/23 16:06:54 Waiting for src bucket item count to get to: 200000 curr count: 112994
2021/12/23 16:06:59 Waiting for src bucket item count to get to: 200000 curr count: 112994
2021/12/23 16:07:04 Waiting for src bucket item count to get to: 200000 curr count: 112994
2021/12/23 16:07:09 Waiting for src bucket item count to get to: 200000 curr count: 129409
2021/12/23 16:07:14 Waiting for src bucket item count to get to: 200000 curr count: 129409
2021/12/23 16:07:19 Waiting for src bucket item count to get to: 200000 curr count: 145687
2021/12/23 16:07:24 Waiting for src bucket item count to get to: 200000 curr count: 145687
2021/12/23 16:07:29 Waiting for src bucket item count to get to: 200000 curr count: 162251
2021/12/23 16:07:34 Waiting for src bucket item count to get to: 200000 curr count: 162251
2021/12/23 16:07:39 Waiting for src bucket item count to get to: 200000 curr count: 177913
2021/12/23 16:07:44 Waiting for src bucket item count to get to: 200000 curr count: 194012
2021/12/23 16:07:49 Waiting for src bucket item count to get to: 200000 curr count: 194012
2021/12/23 16:07:54 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 16:07:54 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 16:07:54 Starting bucket ops to source bucket
2021/12/23 16:11:40 Waiting for dst bucket item count to get to: 0 curr count: 9521
2021/12/23 16:11:45 Waiting for dst bucket item count to get to: 0 curr count: 9521
2021/12/23 16:11:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:11:45 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:11:50 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/23 16:11:55 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/23 16:12:00 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/23 16:12:05 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/23 16:12:05 App: TestKVRebInWithOps got un-deployed
2021/12/23 16:12:05 No eventing-con process running
2021/12/23 16:12:05 Function: TestKVRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps deleting in background"
}
2021/12/23 16:12:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:12:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:12:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:12:13 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:12:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:12:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithOps (449.63s)
=== RUN   TestKVRebOutWithOps
2021/12/23 16:12:23 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:12:24 Removing node(s): [] from the cluster
2021/12/23 16:12:29 Rebalance progress: 100
2021/12/23 16:12:34 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/23 16:12:34 App: TestKVRebOutWithOps got un-deployed
2021/12/23 16:12:34 No eventing-con process running
2021/12/23 16:12:34 Function: TestKVRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithOps not found"
}
2021/12/23 16:12:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:12:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:12:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:12:42 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:12:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:12:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:12:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:12:52 Deploying app: TestKVRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:12:53 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps stored in temp store"
}
2021/12/23 16:12:58 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/23 16:13:03 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/23 16:13:08 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/23 16:13:13 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/23 16:13:13 App: TestKVRebOutWithOps got deployed
2021/12/23 16:13:13 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:13:13 Starting bucket ops to source bucket
2021/12/23 16:13:18 Rebalance progress: 12.5
2021/12/23 16:13:23 Rebalance progress: 12.5
2021/12/23 16:13:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:13:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:13:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:13:29 Waiting for src bucket item count to get to: 200000 curr count: 8423
2021/12/23 16:13:34 Waiting for src bucket item count to get to: 200000 curr count: 24323
2021/12/23 16:13:39 Waiting for src bucket item count to get to: 200000 curr count: 24323
2021/12/23 16:13:44 Waiting for src bucket item count to get to: 200000 curr count: 40727
2021/12/23 16:13:49 Waiting for src bucket item count to get to: 200000 curr count: 40727
2021/12/23 16:13:54 Waiting for src bucket item count to get to: 200000 curr count: 57151
2021/12/23 16:13:59 Waiting for src bucket item count to get to: 200000 curr count: 57151
2021/12/23 16:14:04 Waiting for src bucket item count to get to: 200000 curr count: 73213
2021/12/23 16:14:09 Waiting for src bucket item count to get to: 200000 curr count: 73213
2021/12/23 16:14:14 Waiting for src bucket item count to get to: 200000 curr count: 73213
2021/12/23 16:14:19 Waiting for src bucket item count to get to: 200000 curr count: 88793
2021/12/23 16:14:24 Waiting for src bucket item count to get to: 200000 curr count: 88793
2021/12/23 16:14:29 Waiting for src bucket item count to get to: 200000 curr count: 105260
2021/12/23 16:14:34 Waiting for src bucket item count to get to: 200000 curr count: 105260
2021/12/23 16:14:39 Waiting for src bucket item count to get to: 200000 curr count: 121581
2021/12/23 16:14:44 Waiting for src bucket item count to get to: 200000 curr count: 121581
2021/12/23 16:14:49 Waiting for src bucket item count to get to: 200000 curr count: 137924
2021/12/23 16:14:54 Waiting for src bucket item count to get to: 200000 curr count: 137924
2021/12/23 16:14:59 Waiting for src bucket item count to get to: 200000 curr count: 154393
2021/12/23 16:15:04 Waiting for src bucket item count to get to: 200000 curr count: 154393
2021/12/23 16:15:09 Waiting for src bucket item count to get to: 200000 curr count: 170865
2021/12/23 16:15:14 Waiting for src bucket item count to get to: 200000 curr count: 170865
2021/12/23 16:15:19 Waiting for src bucket item count to get to: 200000 curr count: 186233
2021/12/23 16:15:24 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 16:15:24 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 16:15:24 Starting bucket ops to source bucket
2021/12/23 16:19:09 Waiting for dst bucket item count to get to: 0 curr count: 7989
2021/12/23 16:19:14 Waiting for dst bucket item count to get to: 0 curr count: 7989
2021/12/23 16:19:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:19:14 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:19:19 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/23 16:19:24 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/23 16:19:29 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/23 16:19:34 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/23 16:19:34 App: TestKVRebOutWithOps got un-deployed
2021/12/23 16:19:34 No eventing-con process running
2021/12/23 16:19:34 Function: TestKVRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps deleting in background"
}
2021/12/23 16:19:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:19:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:19:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:19:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:19:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:19:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithOps (449.38s)
=== RUN   TestKVSwapRebWithOps
2021/12/23 16:19:58 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/23 16:19:58 App: TestKVSwapRebWithOps got un-deployed
2021/12/23 16:19:58 No eventing-con process running
2021/12/23 16:19:58 Function: TestKVSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithOps not found"
}
2021/12/23 16:20:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:20:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:20:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:20:06 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:20:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:20:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:20:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:20:16 Deploying app: TestKVSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:20:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps stored in temp store"
}
2021/12/23 16:20:22 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/23 16:20:27 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/23 16:20:32 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/23 16:20:37 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/23 16:20:37 App: TestKVSwapRebWithOps got deployed
2021/12/23 16:20:37 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:20:37 Starting bucket ops to source bucket
2021/12/23 16:20:38 Removing node(s): [] from the cluster
2021/12/23 16:20:43 Rebalance progress: 12.5
2021/12/23 16:20:48 Rebalance progress: 12.5
2021/12/23 16:20:53 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:20:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:20:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:20:53 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 16:20:54 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:20:59 Rebalance progress: 12.5
2021/12/23 16:21:04 Rebalance progress: 12.5
2021/12/23 16:21:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:21:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:21:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:21:09 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 16:21:14 Rebalance progress: 12.5
2021/12/23 16:21:19 Rebalance progress: 12.5
2021/12/23 16:21:24 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:21:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:21:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:21:24 Waiting for src bucket item count to get to: 200000 curr count: 64981
2021/12/23 16:21:29 Waiting for src bucket item count to get to: 200000 curr count: 64981
2021/12/23 16:21:34 Waiting for src bucket item count to get to: 200000 curr count: 80004
2021/12/23 16:21:39 Waiting for src bucket item count to get to: 200000 curr count: 80004
2021/12/23 16:21:44 Waiting for src bucket item count to get to: 200000 curr count: 96214
2021/12/23 16:21:49 Waiting for src bucket item count to get to: 200000 curr count: 96214
2021/12/23 16:21:54 Waiting for src bucket item count to get to: 200000 curr count: 112573
2021/12/23 16:21:59 Waiting for src bucket item count to get to: 200000 curr count: 112573
2021/12/23 16:22:04 Waiting for src bucket item count to get to: 200000 curr count: 129000
2021/12/23 16:22:09 Waiting for src bucket item count to get to: 200000 curr count: 129000
2021/12/23 16:22:14 Waiting for src bucket item count to get to: 200000 curr count: 145228
2021/12/23 16:22:19 Waiting for src bucket item count to get to: 200000 curr count: 145228
2021/12/23 16:22:24 Waiting for src bucket item count to get to: 200000 curr count: 161840
2021/12/23 16:22:29 Waiting for src bucket item count to get to: 200000 curr count: 161840
2021/12/23 16:22:34 Waiting for src bucket item count to get to: 200000 curr count: 177041
2021/12/23 16:22:39 Waiting for src bucket item count to get to: 200000 curr count: 177041
2021/12/23 16:22:44 Waiting for src bucket item count to get to: 200000 curr count: 193081
2021/12/23 16:22:49 Waiting for src bucket item count to get to: 200000 curr count: 193081
2021/12/23 16:22:54 Waiting for src bucket item count to get to: 200000 curr count: 193081
2021/12/23 16:22:59 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/23 16:22:59 Post SET ops. Expected item count: 200000 got 200000
2021/12/23 16:22:59 Starting bucket ops to source bucket
2021/12/23 16:26:44 Waiting for dst bucket item count to get to: 0 curr count: 18836
2021/12/23 16:26:49 Waiting for dst bucket item count to get to: 0 curr count: 257
2021/12/23 16:26:54 Waiting for dst bucket item count to get to: 0 curr count: 257
2021/12/23 16:26:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:26:54 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:27:00 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/23 16:27:05 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/23 16:27:10 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/23 16:27:15 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/23 16:27:15 App: TestKVSwapRebWithOps got un-deployed
2021/12/23 16:27:15 No eventing-con process running
2021/12/23 16:27:15 Function: TestKVSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps deleting in background"
}
2021/12/23 16:27:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:27:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:27:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:27:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:27:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:27:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithOps (460.60s)
=== RUN   TestEventingRebInWithTimers
2021/12/23 16:27:38 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/23 16:27:38 App: TestEventingRebInWithTimers got un-deployed
2021/12/23 16:27:38 No eventing-con process running
2021/12/23 16:27:38 Function: TestEventingRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithTimers not found"
}
2021/12/23 16:27:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:27:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:27:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:27:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:27:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:27:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:27:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:27:57 Deploying app: TestEventingRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:27:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers stored in temp store"
}
2021/12/23 16:28:03 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/23 16:28:08 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/23 16:28:13 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/23 16:28:18 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/23 16:28:18 App: TestEventingRebInWithTimers got deployed
2021/12/23 16:28:18 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 16:28:18 Starting bucket ops to source bucket
2021/12/23 16:28:18 Removing node(s): [] from the cluster
2021/12/23 16:28:23 Rebalance progress: 12.5
2021/12/23 16:28:28 Rebalance progress: 12.5
2021/12/23 16:28:33 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:28:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:28:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:28:33 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:28:38 Rebalance progress: 12.5
2021/12/23 16:28:43 Rebalance progress: 12.5
2021/12/23 16:28:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:28:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:28:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:28:48 Waiting for src bucket item count to get to: 50000 curr count: 27009
2021/12/23 16:28:54 Waiting for src bucket item count to get to: 50000 curr count: 37041
2021/12/23 16:28:59 Waiting for src bucket item count to get to: 50000 curr count: 37041
2021/12/23 16:29:04 Waiting for src bucket item count to get to: 50000 curr count: 49967
2021/12/23 16:29:09 Waiting for src bucket item count to get to: 50000 curr count: 49967
2021/12/23 16:29:19 Waiting for dst bucket item count to get to: 50000 curr count: 45801
2021/12/23 16:29:24 Waiting for dst bucket item count to get to: 50000 curr count: 45801
2021/12/23 16:29:24 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:29:24 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:29:24 Starting bucket ops to source bucket
2021/12/23 16:30:33 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:30:38 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:30:43 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:30:48 Waiting for dst bucket item count to get to: 0 curr count: 41009
2021/12/23 16:30:53 Waiting for dst bucket item count to get to: 0 curr count: 41009
2021/12/23 16:30:58 Waiting for dst bucket item count to get to: 0 curr count: 29993
2021/12/23 16:31:03 Waiting for dst bucket item count to get to: 0 curr count: 29993
2021/12/23 16:31:08 Waiting for dst bucket item count to get to: 0 curr count: 11752
2021/12/23 16:31:13 Waiting for dst bucket item count to get to: 0 curr count: 11752
2021/12/23 16:31:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:31:13 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:31:18 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/23 16:31:23 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/23 16:31:28 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/23 16:31:33 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/23 16:31:33 App: TestEventingRebInWithTimers got un-deployed
2021/12/23 16:31:33 No eventing-con process running
2021/12/23 16:31:33 Function: TestEventingRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers deleting in background"
}
2021/12/23 16:31:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:31:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:31:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:31:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:31:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:31:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithTimers (258.11s)
=== RUN   TestEventingRebOutWithTimers
2021/12/23 16:31:51 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 16:31:52 Removing node(s): [] from the cluster
2021/12/23 16:31:57 Rebalance progress: 100
2021/12/23 16:32:02 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/23 16:32:02 App: TestEventingRebOutWithTimers got un-deployed
2021/12/23 16:32:02 No eventing-con process running
2021/12/23 16:32:02 Function: TestEventingRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithTimers not found"
}
2021/12/23 16:32:03 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:32:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:32:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:32:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:32:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:32:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:32:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:32:20 Deploying app: TestEventingRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:32:21 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers stored in temp store"
}
2021/12/23 16:32:26 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/23 16:32:31 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/23 16:32:36 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/23 16:32:41 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/23 16:32:41 App: TestEventingRebOutWithTimers got deployed
2021/12/23 16:32:41 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:32:41 Starting bucket ops to source bucket
2021/12/23 16:32:46 Rebalance progress: 12.5
2021/12/23 16:32:51 Rebalance progress: 12.5
2021/12/23 16:32:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:32:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:32:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:32:56 Waiting for src bucket item count to get to: 50000 curr count: 10787
2021/12/23 16:33:01 Waiting for src bucket item count to get to: 50000 curr count: 22901
2021/12/23 16:33:06 Waiting for src bucket item count to get to: 50000 curr count: 22901
2021/12/23 16:33:11 Waiting for src bucket item count to get to: 50000 curr count: 34336
2021/12/23 16:33:17 Waiting for src bucket item count to get to: 50000 curr count: 34336
2021/12/23 16:33:22 Waiting for src bucket item count to get to: 50000 curr count: 45030
2021/12/23 16:33:27 Waiting for src bucket item count to get to: 50000 curr count: 45030
2021/12/23 16:33:32 Waiting for src bucket item count to get to: 50000 curr count: 45030
2021/12/23 16:33:42 Waiting for dst bucket item count to get to: 50000 curr count: 41594
2021/12/23 16:33:47 Waiting for dst bucket item count to get to: 50000 curr count: 41594
2021/12/23 16:33:47 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:33:47 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:33:47 Starting bucket ops to source bucket
2021/12/23 16:34:56 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:35:01 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:35:06 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:35:11 Waiting for dst bucket item count to get to: 0 curr count: 45345
2021/12/23 16:35:16 Waiting for dst bucket item count to get to: 0 curr count: 45345
2021/12/23 16:35:21 Waiting for dst bucket item count to get to: 0 curr count: 31220
2021/12/23 16:35:26 Waiting for dst bucket item count to get to: 0 curr count: 31220
2021/12/23 16:35:31 Waiting for dst bucket item count to get to: 0 curr count: 11317
2021/12/23 16:35:36 Waiting for dst bucket item count to get to: 0 curr count: 11317
2021/12/23 16:35:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:35:36 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:35:41 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/23 16:35:46 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/23 16:35:51 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/23 16:35:56 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/23 16:35:56 App: TestEventingRebOutWithTimers got un-deployed
2021/12/23 16:35:56 No eventing-con process running
2021/12/23 16:35:56 Function: TestEventingRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers deleting in background"
}
2021/12/23 16:35:58 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:36:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:36:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:36:05 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:36:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:36:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithTimers (263.41s)
=== RUN   TestEventingSwapRebWithTimers
2021/12/23 16:36:20 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/23 16:36:20 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/23 16:36:20 No eventing-con process running
2021/12/23 16:36:20 Function: TestEventingSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithTimers not found"
}
2021/12/23 16:36:22 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:36:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:36:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:36:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:36:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:36:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:36:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:36:38 Deploying app: TestEventingSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:36:39 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers stored in temp store"
}
2021/12/23 16:36:44 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/23 16:36:49 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/23 16:36:55 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/23 16:37:00 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/23 16:37:00 App: TestEventingSwapRebWithTimers got deployed
2021/12/23 16:37:00 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 16:37:00 Starting bucket ops to source bucket
2021/12/23 16:37:00 Removing node(s): [] from the cluster
2021/12/23 16:37:05 Rebalance progress: 12.5
2021/12/23 16:37:10 Rebalance progress: 12.5
2021/12/23 16:37:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:37:15 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 16:37:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:37:21 Rebalance progress: 12.5
2021/12/23 16:37:26 Rebalance progress: 12.5
2021/12/23 16:37:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:37:31 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 16:37:36 Rebalance progress: 12.5
2021/12/23 16:37:41 Rebalance progress: 12.5
2021/12/23 16:37:46 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:37:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:37:46 Waiting for src bucket item count to get to: 50000 curr count: 45540
2021/12/23 16:37:56 Waiting for dst bucket item count to get to: 50000 curr count: 45824
2021/12/23 16:38:01 Waiting for dst bucket item count to get to: 50000 curr count: 45824
2021/12/23 16:38:01 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:38:01 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:38:01 Starting bucket ops to source bucket
2021/12/23 16:39:11 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:39:16 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:39:21 Waiting for dst bucket item count to get to: 0 curr count: 48235
2021/12/23 16:39:26 Waiting for dst bucket item count to get to: 0 curr count: 48235
2021/12/23 16:39:31 Waiting for dst bucket item count to get to: 0 curr count: 35518
2021/12/23 16:39:36 Waiting for dst bucket item count to get to: 0 curr count: 35518
2021/12/23 16:39:41 Waiting for dst bucket item count to get to: 0 curr count: 15792
2021/12/23 16:39:46 Waiting for dst bucket item count to get to: 0 curr count: 15792
2021/12/23 16:39:51 Waiting for dst bucket item count to get to: 0 curr count: 3615
2021/12/23 16:39:56 Waiting for dst bucket item count to get to: 0 curr count: 3615
2021/12/23 16:39:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:39:56 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:40:01 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/23 16:40:06 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/23 16:40:11 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/23 16:40:16 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/23 16:40:16 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/23 16:40:16 No eventing-con process running
2021/12/23 16:40:16 Function: TestEventingSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers deleting in background"
}
2021/12/23 16:40:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:40:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:40:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:40:25 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:40:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:40:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithTimers (259.93s)
=== RUN   TestKVRebInWithTimers
2021/12/23 16:40:40 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/23 16:40:40 App: TestKVRebInWithTimers got un-deployed
2021/12/23 16:40:40 No eventing-con process running
2021/12/23 16:40:40 Function: TestKVRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithTimers not found"
}
2021/12/23 16:40:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:40:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:40:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:40:48 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:40:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:40:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:40:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:40:58 Deploying app: TestKVRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:40:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers stored in temp store"
}
2021/12/23 16:41:04 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/23 16:41:09 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/23 16:41:14 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/23 16:41:19 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/23 16:41:19 App: TestKVRebInWithTimers got deployed
2021/12/23 16:41:19 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:41:19 Starting bucket ops to source bucket
2021/12/23 16:41:20 Removing node(s): [] from the cluster
2021/12/23 16:41:25 Rebalance progress: 12.5
2021/12/23 16:41:30 Rebalance progress: 12.5
2021/12/23 16:41:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:41:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:41:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:41:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:41:40 Rebalance progress: 12.5
2021/12/23 16:41:45 Rebalance progress: 12.5
2021/12/23 16:41:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:41:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:41:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:41:50 Waiting for src bucket item count to get to: 50000 curr count: 24824
2021/12/23 16:41:55 Waiting for src bucket item count to get to: 50000 curr count: 34375
2021/12/23 16:42:00 Waiting for src bucket item count to get to: 50000 curr count: 34375
2021/12/23 16:42:05 Waiting for src bucket item count to get to: 50000 curr count: 46816
2021/12/23 16:42:10 Waiting for src bucket item count to get to: 50000 curr count: 46816
2021/12/23 16:42:20 Waiting for dst bucket item count to get to: 50000 curr count: 47089
2021/12/23 16:42:20 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:42:20 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:42:20 Starting bucket ops to source bucket
2021/12/23 16:43:30 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:43:35 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:43:40 Waiting for dst bucket item count to get to: 0 curr count: 49277
2021/12/23 16:43:45 Waiting for dst bucket item count to get to: 0 curr count: 49277
2021/12/23 16:43:50 Waiting for dst bucket item count to get to: 0 curr count: 36723
2021/12/23 16:43:55 Waiting for dst bucket item count to get to: 0 curr count: 36723
2021/12/23 16:44:00 Waiting for dst bucket item count to get to: 0 curr count: 15704
2021/12/23 16:44:05 Waiting for dst bucket item count to get to: 0 curr count: 15704
2021/12/23 16:44:10 Waiting for dst bucket item count to get to: 0 curr count: 3555
2021/12/23 16:44:15 Waiting for dst bucket item count to get to: 0 curr count: 3555
2021/12/23 16:44:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:44:15 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:44:20 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/23 16:44:25 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/23 16:44:30 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/23 16:44:35 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/23 16:44:35 App: TestKVRebInWithTimers got un-deployed
2021/12/23 16:44:35 No eventing-con process running
2021/12/23 16:44:35 Function: TestKVRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers deleting in background"
}
2021/12/23 16:44:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:44:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:44:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:44:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:44:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:44:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithTimers (258.76s)
=== RUN   TestKVRebOutWithTimers
2021/12/23 16:44:53 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:44:54 Removing node(s): [] from the cluster
2021/12/23 16:44:59 Rebalance progress: 100
2021/12/23 16:45:04 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/23 16:45:04 App: TestKVRebOutWithTimers got un-deployed
2021/12/23 16:45:04 No eventing-con process running
2021/12/23 16:45:04 Function: TestKVRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithTimers not found"
}
2021/12/23 16:45:06 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:45:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:45:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:45:13 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:45:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:45:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:45:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:45:23 Deploying app: TestKVRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:45:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers stored in temp store"
}
2021/12/23 16:45:29 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/23 16:45:34 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/23 16:45:39 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/23 16:45:44 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/23 16:45:44 App: TestKVRebOutWithTimers got deployed
2021/12/23 16:45:44 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:45:44 Starting bucket ops to source bucket
2021/12/23 16:45:49 Rebalance progress: 12.5
2021/12/23 16:45:54 Rebalance progress: 12.5
2021/12/23 16:45:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:45:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:45:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:45:59 Waiting for src bucket item count to get to: 50000 curr count: 7104
2021/12/23 16:46:04 Waiting for src bucket item count to get to: 50000 curr count: 21874
2021/12/23 16:46:09 Waiting for src bucket item count to get to: 50000 curr count: 21874
2021/12/23 16:46:14 Waiting for src bucket item count to get to: 50000 curr count: 33855
2021/12/23 16:46:19 Waiting for src bucket item count to get to: 50000 curr count: 33855
2021/12/23 16:46:24 Waiting for src bucket item count to get to: 50000 curr count: 44775
2021/12/23 16:46:29 Waiting for src bucket item count to get to: 50000 curr count: 44775
2021/12/23 16:46:34 Waiting for src bucket item count to get to: 50000 curr count: 44775
2021/12/23 16:46:44 Waiting for dst bucket item count to get to: 50000 curr count: 36466
2021/12/23 16:46:49 Waiting for dst bucket item count to get to: 50000 curr count: 36466
2021/12/23 16:46:54 Waiting for dst bucket item count to get to: 50000 curr count: 44958
2021/12/23 16:46:59 Waiting for dst bucket item count to get to: 50000 curr count: 44958
2021/12/23 16:46:59 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:46:59 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:46:59 Starting bucket ops to source bucket
2021/12/23 16:48:08 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:48:13 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:48:18 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:48:23 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:48:28 Waiting for dst bucket item count to get to: 0 curr count: 36018
2021/12/23 16:48:33 Waiting for dst bucket item count to get to: 0 curr count: 36018
2021/12/23 16:48:38 Waiting for dst bucket item count to get to: 0 curr count: 25303
2021/12/23 16:48:43 Waiting for dst bucket item count to get to: 0 curr count: 25303
2021/12/23 16:48:48 Waiting for dst bucket item count to get to: 0 curr count: 8548
2021/12/23 16:48:53 Waiting for dst bucket item count to get to: 0 curr count: 8548
2021/12/23 16:48:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:48:53 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:48:58 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/23 16:49:04 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/23 16:49:09 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/23 16:49:14 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/23 16:49:14 App: TestKVRebOutWithTimers got un-deployed
2021/12/23 16:49:14 No eventing-con process running
2021/12/23 16:49:14 Function: TestKVRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers deleting in background"
}
2021/12/23 16:49:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:49:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:49:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:49:22 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:49:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:49:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithTimers (278.83s)
=== RUN   TestKVSwapRebWithTimers
2021/12/23 16:49:37 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:49:37 App: TestKVSwapRebWithTimers got un-deployed
2021/12/23 16:49:37 No eventing-con process running
2021/12/23 16:49:37 Function: TestKVSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithTimers not found"
}
2021/12/23 16:49:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:49:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:49:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:49:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:49:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:49:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:49:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:49:56 Deploying app: TestKVSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:49:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers stored in temp store"
}
2021/12/23 16:50:02 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/23 16:50:07 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/23 16:50:12 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/23 16:50:17 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/23 16:50:17 App: TestKVSwapRebWithTimers got deployed
2021/12/23 16:50:17 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 16:50:17 Starting bucket ops to source bucket
2021/12/23 16:50:17 Removing node(s): [] from the cluster
2021/12/23 16:50:22 Rebalance progress: 12.5
2021/12/23 16:50:27 Rebalance progress: 12.5
2021/12/23 16:50:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:50:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:50:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:50:33 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 16:50:33 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:50:38 Rebalance progress: 12.5
2021/12/23 16:50:43 Rebalance progress: 12.5
2021/12/23 16:50:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:50:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:50:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:50:48 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 16:50:53 Rebalance progress: 12.5
2021/12/23 16:50:58 Rebalance progress: 12.5
2021/12/23 16:51:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:51:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:51:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:51:08 Waiting for dst bucket item count to get to: 50000 curr count: 30876
2021/12/23 16:51:13 Waiting for dst bucket item count to get to: 50000 curr count: 30876
2021/12/23 16:51:18 Waiting for dst bucket item count to get to: 50000 curr count: 41347
2021/12/23 16:51:23 Waiting for dst bucket item count to get to: 50000 curr count: 41347
2021/12/23 16:51:28 Waiting for dst bucket item count to get to: 50000 curr count: 49740
2021/12/23 16:51:33 Waiting for dst bucket item count to get to: 50000 curr count: 49740
2021/12/23 16:51:33 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:51:33 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:51:33 Starting bucket ops to source bucket
2021/12/23 16:52:43 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:52:48 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:52:53 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:52:58 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:53:03 Waiting for dst bucket item count to get to: 0 curr count: 49071
2021/12/23 16:53:08 Waiting for dst bucket item count to get to: 0 curr count: 49071
2021/12/23 16:53:13 Waiting for dst bucket item count to get to: 0 curr count: 29802
2021/12/23 16:53:18 Waiting for dst bucket item count to get to: 0 curr count: 29802
2021/12/23 16:53:23 Waiting for dst bucket item count to get to: 0 curr count: 16439
2021/12/23 16:53:28 Waiting for dst bucket item count to get to: 0 curr count: 16439
2021/12/23 16:53:33 Waiting for dst bucket item count to get to: 0 curr count: 459
2021/12/23 16:53:38 Waiting for dst bucket item count to get to: 0 curr count: 459
2021/12/23 16:53:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:53:38 Post DELETE ops. Expected item count: 0 got 0
2021/12/23 16:53:43 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:53:48 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:53:53 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:53:59 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:54:04 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:54:09 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/23 16:54:09 App: TestKVSwapRebWithTimers got un-deployed
2021/12/23 16:54:09 No eventing-con process running
2021/12/23 16:54:09 Function: TestKVSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers deleting in background"
}
2021/12/23 16:54:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:54:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:54:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:54:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:54:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:54:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithTimers (294.82s)
=== RUN   TestDuplicateTimerCreationEventingRebIn
2021/12/23 16:54:32 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:54:32 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/23 16:54:32 No eventing-con process running
2021/12/23 16:54:32 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebIn not found"
}
2021/12/23 16:54:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:54:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:54:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:54:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:54:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:54:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:54:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:54:51 Deploying app: TestDuplicateTimerCreationEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:54:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn stored in temp store"
}
2021/12/23 16:54:57 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/23 16:55:02 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/23 16:55:07 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/23 16:55:12 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/23 16:55:12 App: TestDuplicateTimerCreationEventingRebIn got deployed
2021/12/23 16:55:12 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 16:55:12 Starting bucket ops to source bucket
2021/12/23 16:55:13 Removing node(s): [] from the cluster
2021/12/23 16:55:18 Rebalance progress: 12.5
2021/12/23 16:55:23 Rebalance progress: 12.5
2021/12/23 16:55:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:55:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:55:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:55:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:55:33 Rebalance progress: 12.5
2021/12/23 16:55:38 Rebalance progress: 12.5
2021/12/23 16:55:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:55:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:55:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:55:43 Waiting for src bucket item count to get to: 50000 curr count: 34583
2021/12/23 16:55:48 Waiting for src bucket item count to get to: 50000 curr count: 34583
2021/12/23 16:55:53 Waiting for src bucket item count to get to: 50000 curr count: 43324
2021/12/23 16:55:58 Waiting for src bucket item count to get to: 50000 curr count: 43324
2021/12/23 16:56:08 Waiting for dst bucket item count to get to: 50000 curr count: 33469
2021/12/23 16:56:13 Waiting for dst bucket item count to get to: 50000 curr count: 33469
2021/12/23 16:56:18 Waiting for dst bucket item count to get to: 50000 curr count: 47358
2021/12/23 16:56:23 Waiting for dst bucket item count to get to: 50000 curr count: 47358
2021/12/23 16:56:23 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:56:23 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:56:28 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:56:33 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:56:38 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:56:43 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:56:48 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/23 16:56:48 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/23 16:56:48 No eventing-con process running
2021/12/23 16:56:48 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn deleting in background"
}
2021/12/23 16:56:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:56:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:57:00 Waiting for dst bucket item count to get to: 0 curr count: 13986
2021/12/23 16:57:05 Waiting for dst bucket item count to get to: 0 curr count: 13986
2021/12/23 16:57:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:57:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:57:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:57:17 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 16:57:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebIn (174.80s)
=== RUN   TestDuplicateTimerCreationEventingRebOut
2021/12/23 16:57:22 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 16:57:22 Removing node(s): [] from the cluster
2021/12/23 16:57:27 Rebalance progress: 100
2021/12/23 16:57:32 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:57:32 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/23 16:57:32 No eventing-con process running
2021/12/23 16:57:32 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebOut not found"
}
2021/12/23 16:57:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:57:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:57:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 16:57:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:57:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:57:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:57:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 16:57:51 Deploying app: TestDuplicateTimerCreationEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 16:57:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut stored in temp store"
}
2021/12/23 16:57:57 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/23 16:58:02 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/23 16:58:07 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/23 16:58:12 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/23 16:58:12 App: TestDuplicateTimerCreationEventingRebOut got deployed
2021/12/23 16:58:12 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 16:58:12 Starting bucket ops to source bucket
2021/12/23 16:58:17 Rebalance progress: 12.5
2021/12/23 16:58:22 Rebalance progress: 12.5
2021/12/23 16:58:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:58:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:58:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 16:58:27 Waiting for src bucket item count to get to: 50000 curr count: 9183
2021/12/23 16:58:32 Waiting for src bucket item count to get to: 50000 curr count: 23068
2021/12/23 16:58:37 Waiting for src bucket item count to get to: 50000 curr count: 23068
2021/12/23 16:58:42 Waiting for src bucket item count to get to: 50000 curr count: 37035
2021/12/23 16:58:47 Waiting for src bucket item count to get to: 50000 curr count: 37035
2021/12/23 16:58:52 Waiting for src bucket item count to get to: 50000 curr count: 46415
2021/12/23 16:58:57 Waiting for src bucket item count to get to: 50000 curr count: 46415
2021/12/23 16:59:02 Waiting for src bucket item count to get to: 50000 curr count: 46415
2021/12/23 16:59:12 Waiting for dst bucket item count to get to: 50000 curr count: 28778
2021/12/23 16:59:17 Waiting for dst bucket item count to get to: 50000 curr count: 28778
2021/12/23 16:59:22 Waiting for dst bucket item count to get to: 50000 curr count: 44200
2021/12/23 16:59:27 Waiting for dst bucket item count to get to: 50000 curr count: 44200
2021/12/23 16:59:27 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 16:59:27 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 16:59:32 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:59:37 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:59:42 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:59:48 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:59:53 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/23 16:59:53 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/23 16:59:53 No eventing-con process running
2021/12/23 16:59:53 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut deleting in background"
}
2021/12/23 16:59:54 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 16:59:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:00:04 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:00:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:00:06 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:00:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:00:16 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:00:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebOut (179.21s)
=== RUN   TestDuplicateTimerCreationEventingSwapReb
2021/12/23 17:00:26 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:00:26 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/23 17:00:26 No eventing-con process running
2021/12/23 17:00:26 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingSwapReb not found"
}
2021/12/23 17:00:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:00:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:00:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:00:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:00:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:00:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:00:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:00:45 Deploying app: TestDuplicateTimerCreationEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:00:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb stored in temp store"
}
2021/12/23 17:00:51 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/23 17:00:56 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/23 17:01:01 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/23 17:01:06 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/23 17:01:06 App: TestDuplicateTimerCreationEventingSwapReb got deployed
2021/12/23 17:01:06 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 17:01:06 Starting bucket ops to source bucket
2021/12/23 17:01:07 Removing node(s): [] from the cluster
2021/12/23 17:01:12 Rebalance progress: 12.5
2021/12/23 17:01:17 Rebalance progress: 12.5
2021/12/23 17:01:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:01:22 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 17:01:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:01:27 Rebalance progress: 12.5
2021/12/23 17:01:32 Rebalance progress: 12.5
2021/12/23 17:01:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:01:37 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:01:42 Rebalance progress: 12.5
2021/12/23 17:01:47 Rebalance progress: 12.5
2021/12/23 17:01:52 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:01:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:01:52 Waiting for src bucket item count to get to: 50000 curr count: 49643
2021/12/23 17:01:57 Waiting for src bucket item count to get to: 50000 curr count: 49643
2021/12/23 17:02:07 Waiting for dst bucket item count to get to: 50000 curr count: 35397
2021/12/23 17:02:12 Waiting for dst bucket item count to get to: 50000 curr count: 35397
2021/12/23 17:02:17 Waiting for dst bucket item count to get to: 50000 curr count: 46295
2021/12/23 17:02:22 Waiting for dst bucket item count to get to: 50000 curr count: 46295
2021/12/23 17:02:22 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:02:22 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:02:27 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:02:32 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:02:37 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:02:43 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:02:48 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/23 17:02:48 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/23 17:02:48 No eventing-con process running
2021/12/23 17:02:48 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb deleting in background"
}
2021/12/23 17:02:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:02:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:02:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:02:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:03:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:03:06 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:03:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingSwapReb (169.99s)
=== RUN   TestDuplicateTimerCreationKVRebIn
2021/12/23 17:03:16 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:03:16 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/23 17:03:16 No eventing-con process running
2021/12/23 17:03:16 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebIn not found"
}
2021/12/23 17:03:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:03:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:03:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:03:25 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:03:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:03:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:03:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:03:35 Deploying app: TestDuplicateTimerCreationKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:03:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn stored in temp store"
}
2021/12/23 17:03:41 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/23 17:03:46 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/23 17:03:51 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/23 17:03:56 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/23 17:03:56 App: TestDuplicateTimerCreationKVRebIn got deployed
2021/12/23 17:03:56 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:03:56 Starting bucket ops to source bucket
2021/12/23 17:03:56 Removing node(s): [] from the cluster
2021/12/23 17:04:01 Rebalance progress: 12.5
2021/12/23 17:04:06 Rebalance progress: 12.5
2021/12/23 17:04:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:04:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:04:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:04:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:04:16 Rebalance progress: 12.5
2021/12/23 17:04:21 Rebalance progress: 12.5
2021/12/23 17:04:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:04:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:04:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:04:27 Waiting for src bucket item count to get to: 50000 curr count: 31293
2021/12/23 17:04:32 Waiting for src bucket item count to get to: 50000 curr count: 42640
2021/12/23 17:04:37 Waiting for src bucket item count to get to: 50000 curr count: 42640
2021/12/23 17:04:47 Waiting for dst bucket item count to get to: 50000 curr count: 27182
2021/12/23 17:04:52 Waiting for dst bucket item count to get to: 50000 curr count: 27182
2021/12/23 17:04:57 Waiting for dst bucket item count to get to: 50000 curr count: 36616
2021/12/23 17:05:02 Waiting for dst bucket item count to get to: 50000 curr count: 36616
2021/12/23 17:05:07 Waiting for dst bucket item count to get to: 50000 curr count: 48589
2021/12/23 17:05:12 Waiting for dst bucket item count to get to: 50000 curr count: 48589
2021/12/23 17:05:12 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:05:12 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:05:17 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:22 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:27 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:32 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:37 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:42 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/23 17:05:42 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/23 17:05:42 No eventing-con process running
2021/12/23 17:05:42 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn deleting in background"
}
2021/12/23 17:05:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:05:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:05:54 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:05:59 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:05:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:06:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:06:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:06:11 Waiting for dst bucket item count to get to: 0 curr count: 32601
2021/12/23 17:06:16 Waiting for dst bucket item count to get to: 0 curr count: 32601
2021/12/23 17:06:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebIn (189.46s)
=== RUN   TestDuplicateTimerCreationKVRebOut
2021/12/23 17:06:21 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:06:21 Removing node(s): [] from the cluster
2021/12/23 17:06:27 Rebalance progress: 100
2021/12/23 17:06:32 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:06:32 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/23 17:06:32 No eventing-con process running
2021/12/23 17:06:32 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebOut not found"
}
2021/12/23 17:06:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:06:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:06:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:06:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:06:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:06:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:06:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:06:50 Deploying app: TestDuplicateTimerCreationKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:06:51 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut stored in temp store"
}
2021/12/23 17:06:56 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/23 17:07:01 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/23 17:07:06 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/23 17:07:11 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/23 17:07:11 App: TestDuplicateTimerCreationKVRebOut got deployed
2021/12/23 17:07:11 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:07:11 Starting bucket ops to source bucket
2021/12/23 17:07:16 Rebalance progress: 12.5
2021/12/23 17:07:21 Rebalance progress: 12.5
2021/12/23 17:07:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:07:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:07:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:07:26 Waiting for src bucket item count to get to: 50000 curr count: 10325
2021/12/23 17:07:31 Waiting for src bucket item count to get to: 50000 curr count: 24388
2021/12/23 17:07:36 Waiting for src bucket item count to get to: 50000 curr count: 24388
2021/12/23 17:07:41 Waiting for src bucket item count to get to: 50000 curr count: 38067
2021/12/23 17:07:46 Waiting for src bucket item count to get to: 50000 curr count: 38067
2021/12/23 17:07:51 Waiting for src bucket item count to get to: 50000 curr count: 47709
2021/12/23 17:07:56 Waiting for src bucket item count to get to: 50000 curr count: 47709
2021/12/23 17:08:06 Waiting for dst bucket item count to get to: 50000 curr count: 28706
2021/12/23 17:08:11 Waiting for dst bucket item count to get to: 50000 curr count: 28706
2021/12/23 17:08:16 Waiting for dst bucket item count to get to: 50000 curr count: 28706
2021/12/23 17:08:21 Waiting for dst bucket item count to get to: 50000 curr count: 44885
2021/12/23 17:08:26 Waiting for dst bucket item count to get to: 50000 curr count: 44885
2021/12/23 17:08:26 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:08:26 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:08:31 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:37 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:42 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:47 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:52 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:57 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/23 17:08:57 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/23 17:08:57 No eventing-con process running
2021/12/23 17:08:57 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut deleting in background"
}
2021/12/23 17:08:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:09:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:09:09 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:09:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:09:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:09:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:09:20 Waiting for dst bucket item count to get to: 0 curr count: 28876
2021/12/23 17:09:25 Waiting for dst bucket item count to get to: 0 curr count: 28876
2021/12/23 17:09:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebOut (189.90s)
=== RUN   TestDuplicateTimerCreationKVSwapReb
2021/12/23 17:09:35 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:09:36 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/23 17:09:36 No eventing-con process running
2021/12/23 17:09:36 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVSwapReb not found"
}
2021/12/23 17:09:37 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:09:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:09:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:09:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:09:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:09:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:09:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:09:54 Deploying app: TestDuplicateTimerCreationKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:09:55 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb stored in temp store"
}
2021/12/23 17:10:00 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/23 17:10:05 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/23 17:10:10 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/23 17:10:15 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/23 17:10:15 App: TestDuplicateTimerCreationKVSwapReb got deployed
2021/12/23 17:10:15 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:10:15 Starting bucket ops to source bucket
2021/12/23 17:10:15 Removing node(s): [] from the cluster
2021/12/23 17:10:20 Rebalance progress: 12.5
2021/12/23 17:10:25 Rebalance progress: 12.5
2021/12/23 17:10:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:10:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:10:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:10:31 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:10:31 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:10:36 Rebalance progress: 12.5
2021/12/23 17:10:41 Rebalance progress: 12.5
2021/12/23 17:10:46 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:10:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:10:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:10:46 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:10:51 Rebalance progress: 12.5
2021/12/23 17:10:56 Rebalance progress: 12.5
2021/12/23 17:11:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:11:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:11:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:11:01 Waiting for src bucket item count to get to: 50000 curr count: 49684
2021/12/23 17:11:06 Waiting for src bucket item count to get to: 50000 curr count: 49684
2021/12/23 17:11:16 Waiting for dst bucket item count to get to: 50000 curr count: 38886
2021/12/23 17:11:21 Waiting for dst bucket item count to get to: 50000 curr count: 38886
2021/12/23 17:11:21 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:11:21 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:11:26 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:11:31 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:11:36 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:11:41 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:11:47 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/23 17:11:47 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/23 17:11:47 No eventing-con process running
2021/12/23 17:11:47 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb deleting in background"
}
2021/12/23 17:11:48 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:11:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:11:58 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:11:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:12:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:12:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:12:10 Waiting for dst bucket item count to get to: 0 curr count: 13097
2021/12/23 17:12:15 Waiting for dst bucket item count to get to: 0 curr count: 13097
2021/12/23 17:12:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVSwapReb (169.70s)
=== RUN   TestDuplicateTimerFiringEventingRebIn
2021/12/23 17:12:25 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:12:25 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/23 17:12:25 No eventing-con process running
2021/12/23 17:12:25 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebIn not found"
}
2021/12/23 17:12:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:12:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:12:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:12:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:12:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:12:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:12:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:12:44 Deploying app: TestDuplicateTimerFiringEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:12:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn stored in temp store"
}
2021/12/23 17:12:50 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/23 17:12:55 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/23 17:13:00 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/23 17:13:05 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/23 17:13:05 App: TestDuplicateTimerFiringEventingRebIn got deployed
2021/12/23 17:13:05 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 17:13:05 Starting bucket ops to source bucket
2021/12/23 17:13:05 Removing node(s): [] from the cluster
2021/12/23 17:13:10 Rebalance progress: 12.5
2021/12/23 17:13:15 Rebalance progress: 12.5
2021/12/23 17:13:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:13:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:13:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:13:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:13:26 Rebalance progress: 12.5
2021/12/23 17:13:31 Rebalance progress: 12.5
2021/12/23 17:13:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:13:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:13:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:13:36 Waiting for src bucket item count to get to: 50000 curr count: 27911
2021/12/23 17:13:41 Waiting for src bucket item count to get to: 50000 curr count: 39821
2021/12/23 17:13:46 Waiting for src bucket item count to get to: 50000 curr count: 39821
2021/12/23 17:13:56 Waiting for dst bucket item count to get to: 50000 curr count: 37228
2021/12/23 17:14:01 Waiting for dst bucket item count to get to: 50000 curr count: 37228
2021/12/23 17:14:06 Waiting for dst bucket item count to get to: 50000 curr count: 45396
2021/12/23 17:14:11 Waiting for dst bucket item count to get to: 50000 curr count: 45396
2021/12/23 17:14:11 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:14:11 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:14:16 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:14:21 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:14:26 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:14:31 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:14:36 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/23 17:14:36 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/23 17:14:36 No eventing-con process running
2021/12/23 17:14:36 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn deleting in background"
}
2021/12/23 17:14:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:14:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:14:48 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:14:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:14:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:14:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:14:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebIn (159.59s)
=== RUN   TestDuplicateTimerFiringEventingRebOut
2021/12/23 17:15:00 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 17:15:00 Removing node(s): [] from the cluster
2021/12/23 17:15:05 Rebalance progress: 100
2021/12/23 17:15:10 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:15:10 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/23 17:15:10 No eventing-con process running
2021/12/23 17:15:10 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebOut not found"
}
2021/12/23 17:15:12 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:15:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:15:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:15:19 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:15:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:15:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:15:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:15:29 Deploying app: TestDuplicateTimerFiringEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:15:30 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut stored in temp store"
}
2021/12/23 17:15:35 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/23 17:15:40 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/23 17:15:45 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/23 17:15:50 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/23 17:15:50 App: TestDuplicateTimerFiringEventingRebOut got deployed
2021/12/23 17:15:50 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:15:50 Starting bucket ops to source bucket
2021/12/23 17:15:55 Rebalance progress: 12.5
2021/12/23 17:16:00 Rebalance progress: 12.5
2021/12/23 17:16:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:16:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:16:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:16:05 Waiting for src bucket item count to get to: 50000 curr count: 12609
2021/12/23 17:16:10 Waiting for src bucket item count to get to: 50000 curr count: 26775
2021/12/23 17:16:15 Waiting for src bucket item count to get to: 50000 curr count: 26775
2021/12/23 17:16:20 Waiting for src bucket item count to get to: 50000 curr count: 36564
2021/12/23 17:16:25 Waiting for src bucket item count to get to: 50000 curr count: 36564
2021/12/23 17:16:30 Waiting for src bucket item count to get to: 50000 curr count: 47752
2021/12/23 17:16:35 Waiting for src bucket item count to get to: 50000 curr count: 47752
2021/12/23 17:16:40 Waiting for src bucket item count to get to: 50000 curr count: 47752
2021/12/23 17:16:50 Waiting for dst bucket item count to get to: 50000 curr count: 39380
2021/12/23 17:16:55 Waiting for dst bucket item count to get to: 50000 curr count: 39380
2021/12/23 17:16:55 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:16:55 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:17:00 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:05 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:11 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:16 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:21 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:26 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/23 17:17:26 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/23 17:17:26 No eventing-con process running
2021/12/23 17:17:26 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut deleting in background"
}
2021/12/23 17:17:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:17:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:17:38 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:17:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:17:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:17:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:17:49 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:17:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebOut (174.48s)
=== RUN   TestDuplicateTimerFiringEventingSwapReb
2021/12/23 17:17:59 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:17:59 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/23 17:17:59 No eventing-con process running
2021/12/23 17:17:59 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingSwapReb not found"
}
2021/12/23 17:18:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:18:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:18:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:18:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:18:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:18:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:18:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:18:18 Deploying app: TestDuplicateTimerFiringEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:18:19 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb stored in temp store"
}
2021/12/23 17:18:24 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/23 17:18:29 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/23 17:18:34 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/23 17:18:39 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/23 17:18:39 App: TestDuplicateTimerFiringEventingSwapReb got deployed
2021/12/23 17:18:39 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/23 17:18:39 Starting bucket ops to source bucket
2021/12/23 17:18:39 Removing node(s): [] from the cluster
2021/12/23 17:18:44 Rebalance progress: 12.5
2021/12/23 17:18:49 Rebalance progress: 12.5
2021/12/23 17:18:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:18:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:18:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:18:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 17:18:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:19:00 Rebalance progress: 12.5
2021/12/23 17:19:05 Rebalance progress: 12.5
2021/12/23 17:19:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:19:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:19:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:19:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:19:15 Rebalance progress: 12.5
2021/12/23 17:19:20 Rebalance progress: 12.5
2021/12/23 17:19:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:19:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:19:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:19:25 Waiting for src bucket item count to get to: 50000 curr count: 44843
2021/12/23 17:19:30 Waiting for src bucket item count to get to: 50000 curr count: 44843
2021/12/23 17:19:40 Waiting for dst bucket item count to get to: 50000 curr count: 45975
2021/12/23 17:19:45 Waiting for dst bucket item count to get to: 50000 curr count: 45975
2021/12/23 17:19:45 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:19:45 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:19:50 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:19:56 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:20:01 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:20:06 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:20:11 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/23 17:20:11 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/23 17:20:11 No eventing-con process running
2021/12/23 17:20:11 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb deleting in background"
}
2021/12/23 17:20:13 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:20:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:20:23 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:20:28 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:20:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:20:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:20:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:20:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingSwapReb (165.29s)
=== RUN   TestDuplicateTimerFiringKVRebIn
2021/12/23 17:20:45 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:20:45 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/23 17:20:45 No eventing-con process running
2021/12/23 17:20:45 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebIn not found"
}
2021/12/23 17:20:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:20:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:20:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:20:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:20:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:20:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:21:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:21:03 Deploying app: TestDuplicateTimerFiringKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:21:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn stored in temp store"
}
2021/12/23 17:21:09 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/23 17:21:14 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/23 17:21:19 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/23 17:21:24 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/23 17:21:24 App: TestDuplicateTimerFiringKVRebIn got deployed
2021/12/23 17:21:24 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:21:24 Starting bucket ops to source bucket
2021/12/23 17:21:24 Removing node(s): [] from the cluster
2021/12/23 17:21:29 Rebalance progress: 12.5
2021/12/23 17:21:34 Rebalance progress: 12.5
2021/12/23 17:21:39 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:21:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:21:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:21:40 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:21:45 Rebalance progress: 12.5
2021/12/23 17:21:50 Rebalance progress: 12.5
2021/12/23 17:21:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:21:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:21:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:21:55 Waiting for src bucket item count to get to: 50000 curr count: 31171
2021/12/23 17:22:00 Waiting for src bucket item count to get to: 50000 curr count: 31171
2021/12/23 17:22:05 Waiting for src bucket item count to get to: 50000 curr count: 42316
2021/12/23 17:22:10 Waiting for src bucket item count to get to: 50000 curr count: 42316
2021/12/23 17:22:20 Waiting for dst bucket item count to get to: 50000 curr count: 34301
2021/12/23 17:22:25 Waiting for dst bucket item count to get to: 50000 curr count: 34301
2021/12/23 17:22:30 Waiting for dst bucket item count to get to: 50000 curr count: 44387
2021/12/23 17:22:30 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:22:30 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:22:35 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:22:40 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:22:45 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:22:50 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:22:55 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/23 17:22:55 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/23 17:22:55 No eventing-con process running
2021/12/23 17:22:55 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn deleting in background"
}
2021/12/23 17:22:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:23:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:23:07 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:23:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:23:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:23:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:23:19 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:23:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebIn (164.33s)
=== RUN   TestDuplicateTimerFiringKVRebOut
2021/12/23 17:23:24 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:23:24 Removing node(s): [] from the cluster
2021/12/23 17:23:29 Rebalance progress: 100
2021/12/23 17:23:34 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:23:34 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/23 17:23:34 No eventing-con process running
2021/12/23 17:23:34 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebOut not found"
}
2021/12/23 17:23:36 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:23:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:23:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:23:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:23:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:23:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:23:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:23:53 Deploying app: TestDuplicateTimerFiringKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:23:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut stored in temp store"
}
2021/12/23 17:23:59 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/23 17:24:04 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/23 17:24:09 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/23 17:24:14 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/23 17:24:14 App: TestDuplicateTimerFiringKVRebOut got deployed
2021/12/23 17:24:14 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:24:14 Starting bucket ops to source bucket
2021/12/23 17:24:19 Rebalance progress: 12.5
2021/12/23 17:24:24 Rebalance progress: 12.5
2021/12/23 17:24:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:24:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:24:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:24:29 Waiting for src bucket item count to get to: 50000 curr count: 6785
2021/12/23 17:24:34 Waiting for src bucket item count to get to: 50000 curr count: 21088
2021/12/23 17:24:39 Waiting for src bucket item count to get to: 50000 curr count: 21088
2021/12/23 17:24:44 Waiting for src bucket item count to get to: 50000 curr count: 33065
2021/12/23 17:24:49 Waiting for src bucket item count to get to: 50000 curr count: 33065
2021/12/23 17:24:54 Waiting for src bucket item count to get to: 50000 curr count: 44022
2021/12/23 17:24:59 Waiting for src bucket item count to get to: 50000 curr count: 44022
2021/12/23 17:25:04 Waiting for src bucket item count to get to: 50000 curr count: 44022
2021/12/23 17:25:14 Waiting for dst bucket item count to get to: 50000 curr count: 35896
2021/12/23 17:25:19 Waiting for dst bucket item count to get to: 50000 curr count: 35896
2021/12/23 17:25:24 Waiting for dst bucket item count to get to: 50000 curr count: 43759
2021/12/23 17:25:29 Waiting for dst bucket item count to get to: 50000 curr count: 43759
2021/12/23 17:25:29 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:25:29 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:25:34 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:25:39 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:25:44 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:25:50 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:25:55 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:26:00 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/23 17:26:00 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/23 17:26:00 No eventing-con process running
2021/12/23 17:26:00 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut deleting in background"
}
2021/12/23 17:26:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:26:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:26:11 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:26:16 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:26:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:26:18 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:26:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:26:28 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:26:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebOut (189.37s)
=== RUN   TestDuplicateTimerFiringKVSwapReb
2021/12/23 17:26:38 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:26:38 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/23 17:26:38 No eventing-con process running
2021/12/23 17:26:38 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVSwapReb not found"
}
2021/12/23 17:26:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:26:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:26:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:26:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:26:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:26:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:26:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:26:57 Deploying app: TestDuplicateTimerFiringKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:26:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb stored in temp store"
}
2021/12/23 17:27:03 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/23 17:27:08 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/23 17:27:13 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/23 17:27:18 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/23 17:27:18 App: TestDuplicateTimerFiringKVSwapReb got deployed
2021/12/23 17:27:18 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:27:18 Starting bucket ops to source bucket
2021/12/23 17:27:19 Removing node(s): [] from the cluster
2021/12/23 17:27:24 Rebalance progress: 12.5
2021/12/23 17:27:29 Rebalance progress: 12.5
2021/12/23 17:27:34 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:27:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:27:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:27:34 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:27:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:27:40 Rebalance progress: 12.5
2021/12/23 17:27:45 Rebalance progress: 12.5
2021/12/23 17:27:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:27:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:27:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:27:50 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:27:55 Rebalance progress: 12.5
2021/12/23 17:28:00 Rebalance progress: 12.5
2021/12/23 17:28:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:28:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:28:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:28:05 Waiting for src bucket item count to get to: 50000 curr count: 47051
2021/12/23 17:28:10 Waiting for src bucket item count to get to: 50000 curr count: 47051
2021/12/23 17:28:20 Waiting for dst bucket item count to get to: 50000 curr count: 45898
2021/12/23 17:28:25 Waiting for dst bucket item count to get to: 50000 curr count: 45898
2021/12/23 17:28:25 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/23 17:28:25 Post SET ops. Expected item count: 50000 got 50000
2021/12/23 17:28:30 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:28:35 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:28:40 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:28:45 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:28:50 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/23 17:28:50 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/23 17:28:50 No eventing-con process running
2021/12/23 17:28:51 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb deleting in background"
}
2021/12/23 17:28:52 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:28:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:29:02 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:29:07 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:29:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:29:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:29:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:29:19 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/23 17:29:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVSwapReb (170.78s)
=== RUN   TestKVRebOnUpdateBucketOpOneByOne
2021/12/23 17:29:29 Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:29:29 App: bucket_op_on_update got un-deployed
2021/12/23 17:29:29 No eventing-con process running
2021/12/23 17:29:29 Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:29:31 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:29:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:29:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:29:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:29:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:29:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:29:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:29:48 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:29:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:29:54 Starting bucket ops to source bucket
2021/12/23 17:29:59 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:30:04 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:30:09 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:30:09 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:30:09 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:30:10 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:30:15 Rebalance progress: 12.5
2021/12/23 17:30:20 Rebalance progress: 12.5
2021/12/23 17:30:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:30:25 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:30:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:30:30 Rebalance progress: 12.5
2021/12/23 17:30:35 Rebalance progress: 12.5
2021/12/23 17:30:40 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:30:40 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/23 17:30:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:30:46 Rebalance progress: 12.5
2021/12/23 17:30:51 Rebalance progress: 12.5
2021/12/23 17:30:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:30:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:30:56 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:31:01 Rebalance progress: 12.5
2021/12/23 17:31:06 Rebalance progress: 12.5
2021/12/23 17:31:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:31:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:31:16 Rebalance progress: 12.5
2021/12/23 17:31:21 Rebalance progress: 12.5
2021/12/23 17:31:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:31:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 17:31:31 Rebalance progress: 12.5
2021/12/23 17:31:36 Rebalance progress: 12.5
2021/12/23 17:31:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:31:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:31:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:31:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:31:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:32:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:32:07 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:32:07 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:32:07 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:32:07 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:32:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:32:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:32:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10700
2021/12/23 17:32:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:32:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:32:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:32:30 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2090
2021/12/23 17:32:35 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2090
2021/12/23 17:32:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpOneByOne (196.05s)
=== RUN   TestKVForceFailover
2021/12/23 17:32:50 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:32:50 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:32:50 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:32:50 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:32:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:32:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:32:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:32:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:33:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:33:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:33:04 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:33:04 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:33:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:33:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:33:09 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:33:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:33:10 Starting bucket ops to source bucket
2021/12/23 17:33:15 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:33:20 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:33:25 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:33:30 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:33:30 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:33:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:33:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:33:30 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/23 17:33:50 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/23 17:33:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/23 17:33:56 Rebalance progress: 12.5
2021/12/23 17:34:01 Rebalance progress: 12.5
2021/12/23 17:34:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:34:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:34:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:34:06 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/23 17:34:11 Rebalance progress: 12.5
2021/12/23 17:34:16 Rebalance progress: 12.5
2021/12/23 17:34:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:34:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:34:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:34:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:34:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:34:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:34:41 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:34:47 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:34:47 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:34:47 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:34:47 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:34:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:34:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:34:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7068
2021/12/23 17:34:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:35:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:35:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:35:10 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1427
2021/12/23 17:35:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1427
2021/12/23 17:35:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVForceFailover (160.03s)
=== RUN   TestKVHardFailoverRecovery
2021/12/23 17:35:25 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:35:25 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:35:25 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:35:25 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:35:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:35:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:35:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:35:34 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:35:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:35:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:35:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:35:44 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:35:45 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:35:50 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:35:55 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:36:00 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:36:05 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:36:05 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:36:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:36:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:36:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:36:06 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:36:11 Rebalance progress: 12.5
2021/12/23 17:36:16 Rebalance progress: 12.5
2021/12/23 17:36:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:36:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:36:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:36:21 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/23 17:37:21 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/23 17:37:26 Rebalance progress: 12.5
2021/12/23 17:37:31 Rebalance progress: 12.5
2021/12/23 17:37:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:37:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:37:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:37:36 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:37:41 Rebalance progress: 12.5
2021/12/23 17:37:46 Rebalance progress: 12.5
2021/12/23 17:37:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:37:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:37:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:37:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:06 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:11 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:17 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:38:17 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:38:17 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:38:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:38:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:38:28 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12085
2021/12/23 17:38:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:38:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:38:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:38:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1975
2021/12/23 17:38:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1975
2021/12/23 17:38:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVHardFailoverRecovery (209.96s)
=== RUN   TestKVRebOnUpdateBucketOpAllAtOnce
2021/12/23 17:38:55 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:38:55 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:38:55 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:38:55 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:38:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:39:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:39:04 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:39:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:39:14 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:39:15 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:39:20 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:39:25 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:39:30 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:39:35 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:39:35 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:39:35 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:39:35 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:39:35 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/23 17:39:36 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:39:41 Rebalance progress: 12.5
2021/12/23 17:39:46 Rebalance progress: 12.5
2021/12/23 17:39:51 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:39:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:39:51 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/23 17:39:56 Rebalance progress: 12.5
2021/12/23 17:40:01 Rebalance progress: 12.5
2021/12/23 17:40:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:40:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:40:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:40:11 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:40:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:40:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:40:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:40:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:40:31 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:40:31 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:40:31 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:40:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:40:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:40:43 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4612
2021/12/23 17:40:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4612
2021/12/23 17:40:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:40:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:40:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:41:00 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 255
2021/12/23 17:41:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 255
2021/12/23 17:41:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpAllAtOnce (139.90s)
=== RUN   TestKVSwapRebOnUpdateBucketOp
2021/12/23 17:41:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:41:15 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:41:15 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:41:15 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:41:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:41:22 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:41:22 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:41:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:41:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:41:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:41:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:41:29 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:41:30 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:41:35 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:41:40 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:41:45 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:41:50 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:41:50 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:41:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:41:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:41:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:41:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:41:55 Rebalance progress: 12.5
2021/12/23 17:42:00 Rebalance progress: 12.5
2021/12/23 17:42:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:42:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:42:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:42:11 Rebalance progress: 12.5
2021/12/23 17:42:16 Rebalance progress: 12.5
2021/12/23 17:42:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:42:21 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:42:26 Rebalance progress: 12.5
2021/12/23 17:42:31 Rebalance progress: 12.5
2021/12/23 17:42:36 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:42:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:42:41 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:42:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:42:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:42:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:43:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:43:01 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:43:01 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:43:01 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:43:03 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:43:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:43:13 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6089
2021/12/23 17:43:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6089
2021/12/23 17:43:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:43:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:43:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:43:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:43:30 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:43:30 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:43:30 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:43:30 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:43:32 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:43:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:43:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:43:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:43:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:43:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebOnUpdateBucketOp (158.31s)
=== RUN   TestKVRebalanceWithMultipleHandlers
2021/12/23 17:43:53 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:43:53 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:43:53 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:43:53 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:43:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:44:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:44:12 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:44:12 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/23 17:44:12 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:44:12 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/23 17:44:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:19 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:44:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:25 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:44:26 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:44:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:44:26 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:44:28 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/23 17:44:33 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:44:38 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:44:43 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:44:48 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:44:48 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/23 17:44:53 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:44:58 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:45:03 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:45:03 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:03 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:03 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:45:03 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:45:03 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:45:08 Rebalance progress: 12.5
2021/12/23 17:45:13 Rebalance progress: 12.5
2021/12/23 17:45:18 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:45:18 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:45:18 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:45:23 Rebalance progress: 12.5
2021/12/23 17:45:28 Rebalance progress: 12.5
2021/12/23 17:45:33 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:45:33 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/23 17:45:34 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:45:39 Rebalance progress: 12.5
2021/12/23 17:45:44 Rebalance progress: 12.5
2021/12/23 17:45:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:45:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:45:49 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:45:54 Rebalance progress: 12.5
2021/12/23 17:45:59 Rebalance progress: 12.5
2021/12/23 17:46:04 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:46:04 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:46:09 Rebalance progress: 12.5
2021/12/23 17:46:14 Rebalance progress: 12.5
2021/12/23 17:46:19 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:46:19 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/23 17:46:24 Rebalance progress: 12.5
2021/12/23 17:46:29 Rebalance progress: 12.5
2021/12/23 17:46:34 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:46:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:46:39 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:46:45 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:46:50 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:46:55 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:47:00 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:47:05 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:47:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:47:10 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:47:10 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:47:10 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:47:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:47:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:47:23 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12155
2021/12/23 17:47:28 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12155
2021/12/23 17:47:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:47:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:47:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:47:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2631
2021/12/23 17:47:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2631
2021/12/23 17:47:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:47:50 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:47:55 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:48:01 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:48:06 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:48:11 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:48:11 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/23 17:48:11 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:48:11 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/23 17:48:13 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:48:18 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:48:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:48:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:48:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:48:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebalanceWithMultipleHandlers (281.24s)
=== RUN   TestKVSwapRebalanceWithMultipleHandlers
2021/12/23 17:48:35 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:48:35 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:48:35 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:48:35 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:48:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:48:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:48:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:48:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:48:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:48:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:48:53 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:48:53 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/23 17:48:53 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:48:53 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/23 17:48:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:49:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:49:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:49:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:49:07 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:49:08 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:49:08 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:49:09 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/23 17:49:14 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:49:19 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:49:24 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:49:29 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:49:29 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/23 17:49:34 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:49:39 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:49:44 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/23 17:49:44 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:49:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:49:44 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:49:45 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:49:50 Rebalance progress: 12.5
2021/12/23 17:49:55 Rebalance progress: 12.5
2021/12/23 17:50:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:50:00 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:50:00 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:50:05 Rebalance progress: 12.5
2021/12/23 17:50:10 Rebalance progress: 12.5
2021/12/23 17:50:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:50:15 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:50:20 Rebalance progress: 12.5
2021/12/23 17:50:25 Rebalance progress: 12.5
2021/12/23 17:50:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:50:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:50:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:50:41 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:50:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:50:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:50:56 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:50:56 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:50:56 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:50:56 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:51:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:51:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:51:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:51:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:51:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:51:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7635
2021/12/23 17:51:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:51:22 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:51:27 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:51:33 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:51:38 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:51:43 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/23 17:51:43 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/23 17:51:43 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:51:43 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/23 17:51:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:51:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:51:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:51:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:51:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:51:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebalanceWithMultipleHandlers (211.81s)
=== RUN   TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/23 17:52:06 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:52:06 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:52:06 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:52:06 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:52:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:52:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:52:13 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:52:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:52:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:52:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:52:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:52:25 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:52:26 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:52:31 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:52:36 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:52:41 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:52:46 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:52:46 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:52:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:52:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:52:46 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:52:46 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:53:07 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:53:17 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:53:22 Rebalance progress: 12.5
2021/12/23 17:53:27 Rebalance progress: 12.5
2021/12/23 17:53:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:53:32 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:53:33 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:53:38 Rebalance progress: 12.5
2021/12/23 17:53:43 Rebalance progress: 12.5
2021/12/23 17:53:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:53:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:53:48 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:53:53 Rebalance progress: 12.5
2021/12/23 17:53:58 Rebalance progress: 12.5
2021/12/23 17:54:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:54:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:54:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:54:08 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:54:13 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:54:18 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:54:24 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:54:29 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:54:29 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:54:29 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:54:29 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:54:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:54:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:54:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5088
2021/12/23 17:54:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5088
2021/12/23 17:54:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:54:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:54:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:54:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 9187
2021/12/23 17:54:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne (180.95s)
=== RUN   TestKVFailoverOnUpdateBucketOp
2021/12/23 17:55:07 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:55:07 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:55:07 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:55:07 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/23 17:55:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:55:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:55:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:55:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:55:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:55:21 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:55:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:55:26 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:55:27 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:55:32 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/23 17:55:37 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:55:42 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:55:47 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:55:47 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:55:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:55:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:55:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:55:47 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:55:52 Rebalance progress: 12.5
2021/12/23 17:55:57 Rebalance progress: 12.5
2021/12/23 17:56:02 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:56:02 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:56:12 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:56:17 Rebalance progress: 12.5
2021/12/23 17:56:22 Rebalance progress: 12.5
2021/12/23 17:56:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:56:27 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/23 17:56:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/23 17:56:33 Rebalance progress: 12.5
2021/12/23 17:56:38 Rebalance progress: 12.5
2021/12/23 17:56:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:56:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/23 17:56:48 Rebalance progress: 12.5
2021/12/23 17:56:53 Rebalance progress: 12.5
2021/12/23 17:56:58 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:56:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:57:03 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:57:08 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:57:14 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:57:19 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:57:24 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:57:24 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:57:24 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:57:24 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:57:26 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:57:31 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:57:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8622
2021/12/23 17:57:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:57:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:57:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:57:47 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8622
2021/12/23 17:57:47 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVFailoverOnUpdateBucketOp (170.07s)
=== RUN   TestBootstrapAfterKVHardFailover
2021/12/23 17:57:52 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/23 17:57:53 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/23 17:57:58 Rebalance progress: 100
2021/12/23 17:57:58 Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:58:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:58:28 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/23 17:58:29 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/23 17:58:34 Removing node(s): [] from the cluster
2021/12/23 17:58:34 Starting bucket ops to source bucket
2021/12/23 17:58:39 Rebalance failed. See logs for detailed reason. You can try again.
2021/12/23 17:58:44 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:58:49 Waiting for app: bucket_op_on_update to get deployed
2021/12/23 17:58:49 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:58:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:58:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/23 17:58:54 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:58:59 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:59:04 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:59:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:59:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/23 17:59:15 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/23 17:59:15 eventingSetup.go:46: No eventing-con process running
2021/12/23 17:59:15 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/23 17:59:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:59:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:59:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1507
2021/12/23 17:59:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/23 17:59:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/23 17:59:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/23 17:59:38 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1507
2021/12/23 17:59:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBootstrapAfterKVHardFailover (110.88s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	11956.298s

Rebalance tests took 3 hours, 19 minutes and 19 seconds.

Testing Succeeded


Version: versions-23.12.2021-10.21.cfg
Build Log: make-23.12.2021-10.21.log
Server Log: logs-23.12.2021-10.21.tar.gz

Finished