AI Product Tools
Features

Store Index

Learn how to build and manage your product index, the foundation that allows the AI Shop Assistant to understand and recommend your products.

The Store Index is the "brain" of your AI Shop Assistant. It transforms your WooCommerce product catalog into a compact, AI-optimized format. Without a built index, the chatbot cannot provide accurate product recommendations or answer specific questions about your inventory.

Accessing the Store Index

Navigate to: AI Product Tools > Store Index

You can switch between Basic and Advanced views using the toggle in the top-right corner of the screen.

Basic View

The Basic view is designed for quick setup and maintenance. It's perfect for smaller stores or those who want a "set it and forget it" solution.

  • Profile Selector: Choose which indexing profile you want to work with.
  • Enable/Disable: Toggle the indexing system on or off.
  • Build Button: Start the indexing process with a single click.
  • Progress Indicator: Monitor the build status in real-time. You'll see a percentage bar and the number of products processed.
  • Quick Schedule: Set the index to rebuild automatically (Daily, Twice Daily, or Weekly).

Advanced View

The Advanced view provides granular control over what data is indexed and how the process runs. This is recommended for large stores or those with complex product data.

Profile Management

Create multiple profiles for different purposes. For example:

  • Main Catalog: A profile containing all your products.
  • Seasonal Sale: A profile only containing products from a specific category or with a specific tag.
  • B2B Only: A profile for wholesale products.

Field Selection

Choose exactly which product data points the AI should "know" about. Selecting only necessary fields can make the chatbot faster and more accurate.

  • Mandatory Fields: Title, Categories, Price, and Stock Status are always included.
  • Optional Fields:
    • SKU: Helps the bot identify specific products.
    • Full Description: Provides deep context for complex questions.
    • Short Description: Good for quick summaries.
    • Tags: Helps the bot understand product groupings.
    • Attributes: Essential for technical products (e.g., Size, Color, Material, Voltage).

Filter Options

Narrow down which products are included in the index:

  • Categories: Include or exclude specific product categories.
  • Tags: Filter by product tags.
  • Stock Status: Choose to index all products, only those in stock, or only those out of stock.
  • Date Range: Only index products that have been modified within the last X days. This is great for keeping the index fresh without rebuilding everything.
  • Max Products: Set a hard limit on the number of products to index.
  • Filter Mode: Choose whether products must meet All conditions or Any condition to be included.

Global Settings

  • Max Products per Index: The default limit is 10,000 products. If you have more, you may need to use multiple profiles.
  • Batch Size: Control how many products are processed at once (50-500). Smaller batches are safer for low-resource servers.
  • Scheduled Rebuilds: Enable automatic updates and set the frequency.

Build Controls

  • Build: Start an incremental build. This only processes products that have been added or changed since the last build.
  • Force Rebuild: Wipe the existing index and start from scratch. Use this if you've made major changes to your product structure.
  • Cancel: Stop a running build process safely.
  • Reset: Clear all indexed data for the current profile.

How It Works

The indexing process follows a sophisticated workflow to ensure performance and accuracy:

  1. Conversion: Your product data is converted into a compact format called TOON. This format is designed to be extremely token-efficient, allowing the AI to "read" more products in a single request.
  2. Sharding: The index is split into manageable "shards" based on categories.
  3. Context Injection: When a customer asks a question, the chatbot analyzes the query and only loads the relevant shards. For example, if a customer asks about "shoes," the bot only loads the "Footwear" shard. This keeps the AI responses fast and accurate.
  4. Background Processing: The index is built in the background using the Action Scheduler. This ensures that your store's frontend performance is never compromised during the build process.

Troubleshooting

Build is Stuck

  1. Check if Action Scheduler is running correctly on your site.
  2. Ensure you have enough disk space on your server.
  3. Try reducing the Batch Size in the Advanced settings.
  4. Check your server's error logs for any PHP memory limit issues.

Products are Missing from Chat

  1. Verify that the missing products are included in your active Index Profile.
  2. Check the Filter Options to ensure the products aren't being excluded by category or stock status.
  3. Run a Force Rebuild to ensure the index is fully synchronized with your WooCommerce catalog.

Frequently Asked Questions

How often should I rebuild the index?

For most stores, a daily rebuild is sufficient. If you add new products or change prices multiple times a day, consider the "Twice Daily" schedule.

Does indexing use my AI credits?

No. Building the index is a local process on your server and does not use any AI credits. Credits are only used when the chatbot generates a response for a customer.

Can I index custom fields?

Yes. In the Pro version, you can select custom meta fields to be included in the index, allowing the bot to answer questions about specialized product data.

Building the index for the first time may take several minutes depending on the size of your catalog. Subsequent builds will be much faster as they only process changes.

If you have a very large catalog (over 5,000 products), consider creating separate profiles for major categories to improve the chatbot's response accuracy.

Scheduled rebuilds require WP-Cron to be functioning correctly. If you have disabled WP-Cron, you must set up a server-level cron job to ensure your index stays up-to-date.

Real-World Use Cases

Large Catalogs (10,000+ Products)

For massive stores, building a single index can be resource-intensive. Use the Advanced View to create separate profiles for your main categories (e.g., "Electronics," "Clothing," "Home & Garden"). This makes the indexing process faster and helps the chatbot find products more accurately.

Flash Sales and Promotions

If you're running a 24-hour flash sale, create a profile that only includes products with the "Sale" tag. Set the chatbot to use this profile during the sale so it prioritizes discounted items in its recommendations.

Technical Support Bot

Create a profile that includes all technical attributes and long descriptions. This allows the chatbot to act as a technical support assistant, answering complex questions about product specifications and compatibility.

Pro Tips for Indexing Success

  • Exclude Out-of-Stock Items: To provide the best customer experience, use the filter options to exclude products that are out of stock. This prevents the chatbot from recommending items that customers can't buy.
  • Optimize Your Batch Size: If you're on a shared hosting plan, set your batch size to 50. If you're on a powerful VPS or Dedicated server, you can increase it to 500 for much faster builds.
  • Use "Force Rebuild" Sparingly: Incremental builds are much faster and use fewer server resources. Only use "Force Rebuild" if you've made significant changes to your product attributes or category structure.
  • Monitor Build Status: Keep an eye on the "Last Build Time" in the status panel. If the index hasn't been updated in several days, check your WP-Cron settings.

Technical Details

The Store Index uses a custom-built TOON (Tabular Object-Oriented Notation) format. This format was developed specifically for this plugin to solve the problem of LLM context limits. By stripping away unnecessary JSON syntax and using abbreviated keys, we can fit 5-10 times more product data into a single AI request compared to standard formats.

The indexing engine uses Keyset Pagination (ID > last_id) instead of traditional offset pagination. This makes the process significantly faster and more reliable for large databases, as it avoids the performance degradation that occurs with large OFFSET values in SQL.

Data is stored in sharded .toon.gz files within your WordPress uploads directory. These files are compressed to save disk space and are protected from direct public access. The ShardManager handles the logic of loading and unloading these shards into the AI's memory as needed during a chat session.

Glossary of Terms

  • Indexing: The process of collecting, parsing, and storing data to facilitate fast and accurate information retrieval.
  • TOON (Tabular Object-Oriented Notation): Our custom, token-efficient data format designed specifically for AI processing.
  • Sharding: The process of breaking a large database into smaller, faster, more easily managed parts called shards.
  • Incremental Build: A build process that only updates the parts of the index that have changed since the last build.
  • Force Rebuild: A process that deletes the entire index and recreates it from scratch.
  • Keyset Pagination: A modern method of navigating through large datasets that is much faster than traditional page-based navigation.

Conclusion

A well-maintained Store Index is the foundation of a successful AI-powered store. It ensures that your AI Shop Assistant has the most accurate and up-to-date information about your products, allowing it to provide the best possible recommendations to your customers.

By understanding the different view modes, filtering options, and build controls, you can optimize your indexing process to suit your store's specific needs. Whether you have a handful of products or a massive catalog, the Store Index provides the performance and reliability needed to power a truly intelligent shopping experience.

Need Help?

If you encounter any issues while using the Store Index or have questions about the best settings for your store, our support team is here to help.

  • Documentation: Visit our Help Center for more guides.
  • Support Ticket: Open a ticket through the Support page in the plugin.
  • Community: Join our community forum to share tips and tricks with other store owners.

Data Freshness and Accuracy

The accuracy of your AI Shop Assistant depends entirely on the freshness of your Store Index. If you change a product's price or stock status in WooCommerce, the chatbot will not know about the change until the index is rebuilt.

  • Real-time Updates: While we don't currently support real-time indexing for every single change (to protect your server's performance), the incremental build process is fast enough to run several times a day.
  • Manual Sync: If you've just made a critical change to a top-selling product, we recommend running a manual "Build" from the Basic view to ensure the chatbot is immediately updated.
  • Stock Status: The indexing engine is particularly sensitive to stock status. If a product goes out of stock, it will remain in the index until the next rebuild, but the chatbot is instructed to check the live stock status before finalizing a recommendation whenever possible.

Maintenance and Updates

To keep your AI Shop Assistant performing at its best, we recommend a simple maintenance routine:

  • Weekly Review: Check the "Last Build Time" once a week to ensure your scheduled rebuilds are running as expected.
  • Profile Audit: Every few months, review your indexing profiles. If you've added new categories or attributes to your store, ensure they are included in your active profiles.
  • Clean Up Old Profiles: Delete any indexing profiles that you are no longer using to keep your dashboard clean and organized.
  • Monitor Disk Space: While TOON files are small, a very large catalog with many profiles can eventually take up significant disk space. Periodically check your wp-content/uploads/aipt-index/ folder.

Performance Optimization

For stores with very large catalogs, indexing performance is key. Here are some ways to optimize your build:

  • Increase PHP Memory Limit: Indexing can be memory-intensive. Ensure your server has at least 512MB of PHP memory allocated.
  • Use a Persistent Object Cache: Plugins like Redis or Memcached can speed up the database queries needed to build the index.
  • Optimize Your Database: Regularly clean up your WooCommerce database (e.g., removing old revisions and transients) to ensure the indexing engine can find products quickly.
  • Limit Optional Fields: If you don't need the chatbot to know about every single product attribute, uncheck the ones you don't use. This reduces the size of the index and makes the chatbot faster.

Security and Privacy

We take the security of your store data seriously. The Store Indexing process is designed with privacy in mind:

  • Local Storage: Your product index is stored locally on your own server. It is never uploaded to our servers or shared with third parties.
  • Encrypted Transmission: When the chatbot communicates with an AI provider, only the necessary product data for that specific conversation is sent over a secure, encrypted connection.
  • Access Control: The index files are stored in a protected directory within your WordPress uploads folder, preventing direct access from the web.
  • Data Minimization: You have full control over which fields are indexed. If you have sensitive internal data in your product descriptions, you can simply exclude those fields from the index.

What's Next?

AI Shop Assistant

Enhance your customer experience with an AI-powered chatbot that helps shoppers find products, answers questions, and provides personalized recommendations.

API Configuration

Learn how to connect your preferred AI providers to AI Product Tools.