Discovery Pipeline Control Center
Configure each stage · Set parameters · Review the run plan · Execute sequentially with live console output
Analyzing
3
Ready for pipeline
Approved
0
Scored · awaiting sync
Active Products
82
Approved + synced
Suppliers
207
Alibaba current
Vendors
383
Scored in system
TikTok Data
28
Current P11 records
Instagram Data
28
Current P12 records
1
Pipeline stages
Toggle stages on/off · click to configure below
Amazon
Discovery
Discovery
ACTIVE
→
Amazon
Enrichment
Enrichment
ACTIVE
→
Google
Trends
Trends
ACTIVE
→
Alibaba
Sourcing
Sourcing
ACTIVE
→
→
Score
Calculation
Calculation
ACTIVE
→
JAMMA
Sync
Sync
SKIP
→
Price
Intelligence
Intelligence
SKIP
← Scroll horizontally if needed · Click any stage card to toggle · Arrow color shows active pipeline path →
1b
8-Profile Run Mode
Score Calc V2.6 reads these to apply the correct pillar weights
Discovery Profile + Region
pipeline_region · pipeline_run_channel · pipeline_run_buyer · pipeline_run_product_type
Region [V8.17]
🇦🇪 UAE
🇸🇦 KSA
🌐 Both
Channel
GP
Catalogue
Both
Buyer Type
B2C
B2B
Both
Product Type
Durable
Consumable
Auto
Loading profile weights...
2
Stage configuration
Expand each panel to set parameters
🔍 Amazon Discovery
ACTIVE
▼
Batch size (discovery_batch_size · 1–200)
categories per run
Discovery countries
✓
United States✓
UAE✓
Canada✓
Saudi ArabiaActive platforms (x_config_platform)
✓
Amazon US
✓
Amazon AE
✓
Amazon CA
✓
Amazon SA
Managed in x_config_platform
Search strategies (x_config_strategy)
✓
Strategy 5: Generic Category Only
✓
Strategy 4: Feature-Based Search
✓
Strategy 3: Material-Based Search
✓
Strategy 2: Brand Removal (Category Only)
✓
Strategy 1: Exact Match (Brand + Category)
Brand filter (x_config_brand)
All brands
Western only
Chinese only
Exclude branded
Product categories (product.category)
Shows only discovery targets (x_is_discovery_target=True). Expand parent groups to pick subcategories. Leave empty = all discovery targets.
V8.12 — Embedded blocks (run before category filter)
Block 0 — Pipeline Reset
V8.11
Layer 1: re-enables Sync-auto-disabled categories before batching. Layer 2: TTL-resets stale products (rejected=14d, analyzing=30d, approved=60d, synced=90d) — NULL-date products caught with OR condition. Toggle:
reset_categories_each_run.
Block 0B — Desertcart Stage 0 Discovery
V8.12
For each discovery target category (up to
dc_discovery_batch_size), queries SerpAPI site:desertcart.ae {keywords}. Extracts ASIN (B+9 alphanum) from title/snippet/link. Falls back to Amazon keyword search when no ASIN found. Skips existing ASINs. Creates x_product_core stubs (status=analyzing) + x_desertcart_data records. Toggle: dc_discovery_enabled.Block 0 output → Technical → Logging (path=pipeline_reset). Block 0B output → ir.logging (path=dc_discovery, func=block_0b_v8_12).
Min price (AED)
Max price (AED)
🔬 Amazon Enrichment
ACTIVE
▼
Target products
Analyzing only
All unsynced
Manual selection
Refresh mode
Stale > 30 days
Force all
BSR history snapshot (x_api_data)
✓
Record BSR on each run (time-series)
Builds trend comparison across enrichment runs
Data source platform
Match product country
Force AE
Force US
📈 Google Trends V3.9.0
ACTIVE
▼
Pipeline blocks — V3.9.0 (8 blocks, run in sequence)
Block A
Google Trends — SerpAPI interest score, direction (Rising/Stable/Declining), volatility, regional interest, related queries. Cache TTL:
trends_refresh_days.
Block B
Google Shopping UAE — SerpAPI
google_shopping gl=ae. Reverse-priority query (brand+name → brand+cat → name → ASIN → cat). Writes x_google_shopping (x_min_price_aed, x_result_count, x_gap_signal, x_gap_pts 0–3). Gap pts feed P13 Gate 1 boost. Cache TTL: shopping_refresh_days.
Block C
Gate 1 — Profile-aware threshold (P13 + Shopping gap_pts recapped at 15, P15 ≥ profile_min). Products that pass are forwarded to Blocks D–H. Products that fail are rejected.
Block D
Desertcart UAE Presence — SerpAPI
site:desertcart.ae {ASIN}. Writes x_desertcart_data (import_frequency, x_price_aed, x_gap_pts 0–3). Sets x_has_desertcart_data. Feeds P13 cluster cap in Score Calc.
Block E
Gate G1 (Cross-source) — BSR < 100k AND (Shopping gap ≥ 1 OR Desertcart active OR Trends > 50 + listings < 5). G1 passers forwarded to Blocks F, G, H and then to Alibaba. x_gate_reached = gate_g1.
Block F
Noon UAE Presence — SerpAPI
site:noon.com {ASIN}. Writes x_noon_data (x_result_count, x_min_price_aed, x_avg_rating, x_gap_pts). Sets x_has_noon_data. Feeds P14 Noon sub-signals + P06 Noon floor in Score Calc. Cache TTL: noon_refresh_days.
Block G
EcomHunt Demand — SerpAPI
site:ecomhunt.com {product_name[:80]} gl=us. Runs on ALL Gate 1 passers (not just G1). Detects listing presence and winning/viral keywords in snippets. Writes x_ecomhunt_data. Sets x_has_ecomhunt_data. Feeds P02 (+3 listed, +4 winning) in Score Calc. Toggle: ecomhunt_enabled.
Block H
TikTok Ads Intelligence — RapidAPI
GET /ads/top_products/ region=AE. One call per unique x_category (cached in memory). Fuzzy 40-char product name match. Writes x_tiktok_ads_match, x_tiktok_ad_engagement, x_tiktok_ad_videos to x_product_core. All profiles. Feeds P02 (+4 match, +2 engagement ≥ 5%) in Score Calc. Toggle: tiktok_ads_enabled.
Trends cache (trends_refresh_days)
7 days
14 days
30 days
Shopping cache (shopping_refresh_days)
7 days
14 days
30 days
Noon cache (noon_refresh_days)
7 days
14 days
30 days
Trends force refresh (trends_force_refresh)
Use cache
Force refresh
Shopping force refresh (shopping_force_refresh)
Use cache
Force refresh
Noon force refresh (noon_force_refresh)
Use cache
Force refresh
EcomHunt Block G (ecomhunt_enabled)
Enabled
Disabled
Block G — SerpAPI site:ecomhunt.com · All Gate 1 passers
TikTok Ads Block H (tiktok_ads_enabled)
Enabled
Disabled
Block H — TikTok Ads Top Products AE · All profiles
TikTok Ads period (tiktok_ads_period)
1 day
7 days
30 days
Rolling window for TikTok Ads Top Products API
🏭 Alibaba Sourcing
ACTIVE
▼
Active platforms (x_config_platform)
✓
Platform: Alibaba.com (Global)
✓
Platform 1 (1688)
Suppliers per product (alibaba_suppliers_per_product)
1
2
3
5
10
Min DSR score (vendor_sync_min_dsr)
0 = accept all DSR scores
Min years active
Any
1+
3+
5+
Min 30-day sales (vendor_sync_min_sales30d)
Any
100+
500+
1,000+
Supplier quality filters
✓
Gold suppliers only
✓
Trade assurance required
✓
Clean search query (remove double spaces)
✓
Strip western brand from query
External API data (x_api_data)
✓
Write supplier count back to product core
✓
Auto-batch mode (process all selecting)
📱 Social Data V3.0 — TikTok & Instagram
ACTIVE
▼
Profile scope: Social only runs for GP-B2C-DUR and GP-B2C-CON profiles. All other profiles skip Social entirely (x_social_tiktok / x_social_instagram flags on x_config_profile_weights). x_social_attempted is set True before the first API call so Score Calc can distinguish "skipped by design" from "attempted but no data".
API configuration
TikTok Search
500 calls/day · $0.004/req · tiktok-scraper7.p.rapidapi.com · GET /feed/search
TikTok Ads
Same key · GET /ads/top_products/ (Block H in Trends V3.9.0 — not this stage)
Instagram
20 calls/day · $0.004/req · instagram-scraper-stable-api.p.rapidapi.com
Manage API keys and rate limits in Settings → x_config_platform.
TikTok Search Video — V3.0 extended signals (P11, max 20 pts)
Base signals
Total views, video count, avg engagement rate, content velocity 7d — same as V2.11.
Organic vs Ad split
Reads
is_ad per video. Organic > Ad → +4 P11. Ad count > 0 → +2 P11. Writes x_organic_count, x_ad_count.
Saves (purchase intent)
Reads
collect_count per video. Sum ≥ 10,000 → +4 P11. Sum ≥ 1,000 → +2 P11. Writes x_collect_count.
Recency
Reads
create_time per video. Most recent < 7d → +5. < 30d → +3. < 90d → +1. Writes x_most_recent_date. Cap: 20 pts total.
Instagram hashtag cascade — V2.2 (4 candidates, best postsCount wins)
Candidate 1product words [0:2] — most specific pair
Candidate 2product words [1:3]
Candidate 3product word [2] alone
Candidate 4category map hashtag (e.g. aquariumlife, petcare)
Selection ruleHighest postsCount wins. Batch size + min posts threshold set in ir.config_parameter.
🎯 Score Calculation
ACTIVE
▼
Scoring ruleset (x_config_scoring)
JAMMA Default Scoring Rules
JSIS scoring structure — V2.13 (max 216 pts · 8-Profile weighted)
| P01 Eligibility | max 20 | Price AED range, rating ≥ 4.0, reviews, fragile flag, durable |
| P02 Demand | max 20 | BSR rank + review volume + Trends · +EcomHunt (+3 listed, +4 winning) · +TikTok Ads (+4 match, +2 engagement ≥ 5%) |
| P03 Logistics | max 13 | Fragility, price band, durable, on-time ship |
| P04 Flexibility | max 10 | Easy to explain, family use, price band |
| P05 Coordination | max 5 | Has Alibaba supplier, has trends data, trade assurance |
| P06 Margin | max 24 | Alibaba cost vs UAE retail floor — chain: Google Shopping AED → Noon AED → Amazon AED fallback |
| P07 Quality | max 16 | Amazon rating depth, verified reviews, gold/verified supplier |
| P08 Scalability | max 10 | Alibaba 30d sales volume, trade assurance, supplier count |
| P09 Social/Trends | max 10 | Google Trends score, direction (Rising/Stable/Declining), sustained interest |
| P10 Velocity | max 13 | BSR rank delta across two enrichment runs (x_api_data) + Alibaba 30d sales trend |
| P11 TikTok Social | max 20 | Views + engagement · V3.0: organic vs ad split, saves (collect_count), recency signal · GP-B2C only |
| P12 Instagram | max 10 | Best hashtag postsCount, engagement rate · GP-B2C only |
| P13 Market Gap | max 15 | Google Shopping gap_pts + Desertcart cluster (UAE_gap+8, ShopLow+3, DC_active+3, DC_premium+3) — Noon guard applied |
| P14 Competition | max 10 | Noon UAE listing count sub-signals (0→+8, 1-3→+4, 4-5→+2, no-express+4, rating<4→+4) · fallback: BSR/review barrier |
| P15 Demand Conf | max 10 | Google Trends score + TikTok cross-signal confirmation |
| P_Cross Validation | max 10 | PRIORITY_LAUNCH=10 / NORMAL_LAUNCH=5 / WEAK_SIGNAL=2 / DO_NOT_LAUNCH=0 |
| JSIS MAX | 216 | 8-Profile weighted · GP-B2C-DUR/CON=216 · GP-B2B-DUR=176 · GP-B2B-CON=166 · CAT-B2C=176 · CAT-B2B-CON=156 |
Cross-Platform gate: P11 < 5 AND P12 < 3 forces Grade F regardless of JSIS total. Conflict flags (A–G): priority / hold / reject written per product — reject blocks approval. Profile approval threshold = x_approval_pct × profile ceiling (e.g. 37% × 216 = 80 pts for GP-B2C).
Manage rulesets in x_config_scoring
Batch size (score_batch_size)
50
200
500
Approval threshold pts / 216 JSIS max
70 Relaxed
80 Standard
85 Default
95 Strict
110 Premium
Current config: Approval=85 | Rejection=40 | Min P10=3 | Min Price=37 AED | Max Price=1850 AED | Min Rating=4.0 | Min Reviews=100 (from JAMMA Default Scoring Rules)
Min P10 velocity gate
3 — Relaxed
4 — Moderate
5 — Strict
Rejection threshold
40
50
60
Min rating
3.5 ★
4.0 ★
4.5 ★
Min reviews
50
100
200
500
Gate defaults
✓
Enable gate defaults
✓
Exclude synced products
🔁 JAMMA Sync
SKIPPED
▼
Sync phases
✓
Phase 1: Vendor sync → res.partner
✓
Phase 2: Product sync → product.template
Team price discount
20%
25%
30%
Currency rates (ir.config_parameter)
USD → AED
CAD → AED
SAR → AED
GBP → AED
🧠 Category Intelligence
Phase E
▼
Light route above (router_light_above)
5.0
6.0
7.0
8.0
9.0
Composite score >= this → Light route
Deep route below (router_deep_below)
2.0
3.0
4.0
5.0
6.0
Composite score <= this → Deep route
New category default (router_new_category_default)
light
medium
deep
0-run categories fall back to this route
Auto-disable dead categories (auto_disable_enabled)
Enabled
Disabled
0% yield + JSIS < 80 after min-runs → sets x_is_discovery_target = False
Min runs before auto-disable (auto_disable_min_runs)
1
2
3
5
Category must fail at least this many runs before being disabled
💰 Cost & Budget
Phase F
▼
Daily API budget (USD) (pipeline_daily_budget_usd)
$15
$25
$50
$100
Shared across all pipeline stages per calendar day (UTC)
Enrichment cache (days) (enrichment_refresh_days)
1 day
3 days
7 days
14 days
Amazon enrichment data freshness window
Social cache (days) (social_refresh_days)
3 days
7 days
14 days
TikTok + Instagram data freshness window
Alibaba cache (days) (alibaba_refresh_days)
7 days
14 days
30 days
Alibaba supplier data freshness window
3
Run preview
Updates live as you configure above
📋 Pipeline Run Summary
Updates live as you configure stages
▼
Discovery
→
Enrichment
→
Trends
→
Alibaba
→
TikTok & Insta
→
Score Calc
| Products in queue | 3 (analyzing status) |
| Batch size | 100 per run |
| Countries | CN, US, AE, CA, SA, CN |
| Suppliers / product | 3 |
| Approval gate | 80 pts · P10 ≥ 3 |
| JSIS max score | 216 pts (P01-P10=141 + P11=20 + P12=10 + P13=15 + P14=10 + P15=10 + Cross=10) |
| Stages active | 5 of 7 |
| Execution mode | Sequential (one stage completes before next begins) |
3b
ASIN Filter — optional · collapsed by default
Paste specific ASINs to target instead of full auto-batch
🎯 ASIN Product Filter
Leave empty = auto-batch mode
▼
Today's products — click row to add ASIN to filter
Select all
Clear filter
No products updated today yet.
Selected ASIN filter (leave empty for auto-batch mode)
When filled: pipeline runs only on these products — stored in
pipeline_filter_asins.
Leave empty = normal auto-batch.
✕ Clear
📋 Paste clipboard
4
Execute
Sequential run · console output below
Ready to run
Amazon Discovery
Waiting
Amazon Enrichment
Waiting
Google Trends
Waiting
Alibaba Sourcing
Waiting
TikTok & Instagram
Waiting
Score Calculation
Waiting
JAMMA Sync
Skipped
Price Intelligence
Skipped
▶ Execution console
────────────────────────────────────────────────────────────────
JAMMA Discovery Pipeline Control v1.0
────────────────────────────────────────────────────────────────
> Pipeline ready. Configure stages above then click Run Pipeline.
> Stages will execute sequentially. Each stage must complete before the next begins.
> Console output from each script will appear here in real time.
5
Last Pipeline Runs — today only
Collapsed by default · expand to review run history
📊 Today's Pipeline Runs
Updates on page reload
▼
No pipeline runs today yet.