41 Commits

Author SHA1 Message Date
544f789e2e aa 2025-04-04 16:02:11 -04:00
c1ab5d611f a 2025-04-04 15:33:47 -04:00
c9b57f00ea asdf 2025-04-04 15:31:06 -04:00
45c589d225 b 2025-04-04 15:10:39 -04:00
60eee07249 a 2025-04-04 14:54:51 -04:00
3d95553cbd a 2025-04-02 08:17:13 -04:00
2d01dae9ed a 2025-03-28 11:51:57 -04:00
1983f19fa7 a 2025-03-28 10:46:51 -04:00
8a26abc5f4 a 2025-03-24 15:31:38 -04:00
d76258eb55 asdf 2025-03-18 12:38:30 -04:00
86498d54b4 agg add file, skip foil api pricing, update pricing algo 2025-03-18 12:11:39 -04:00
2800135375 a 2025-02-28 18:41:35 -05:00
c3b4fe28d2 a 2025-02-28 18:38:50 -05:00
d4ffe180a3 a 2025-02-28 18:38:26 -05:00
8b0396ba00 a 2025-02-28 10:12:49 -05:00
99dfc3f6f8 asdf 2025-02-28 10:09:29 -05:00
8a4ec31bee 123 2025-02-28 10:04:24 -05:00
cafc77d07f asdf 2025-02-28 10:04:07 -05:00
dff5bc4a23 k 2025-02-27 13:29:41 -05:00
8097fba83c a 2025-02-27 12:43:05 -05:00
da492180b4 api pricing 2025-02-27 12:37:02 -05:00
e13b871fda asdf 2025-02-21 12:28:08 -05:00
ac6397de01 asdf 2025-02-21 12:26:28 -05:00
4c6d256316 order fix 2025-02-21 12:25:02 -05:00
1bf255d0fe orders 2025-02-21 11:46:50 -05:00
721b26ce97 s 2025-02-16 22:49:18 -05:00
92d1356c0e 5 2025-02-14 23:32:20 -05:00
3f8a99b61a 4 2025-02-14 23:16:48 -05:00
012bb40a04 3 2025-02-14 23:12:06 -05:00
85329c232c 2 2025-02-14 23:07:09 -05:00
7c29e2d8d7 1 2025-02-14 23:01:22 -05:00
cc315129b9 i 2025-02-14 22:45:47 -05:00
511d4dbcee price 2025-02-12 21:04:53 -05:00
af0e789ec9 fix 2025-02-11 19:33:47 -05:00
fbd6dd5752 fixy 2025-02-11 19:30:20 -05:00
aa1cdc2fb3 idk 2025-02-10 20:51:20 -05:00
c7686fb239 open box bug 2025-02-10 20:35:26 -05:00
c896a6ea0f inventory bug 2025-02-10 20:12:20 -05:00
edf76708b3 box bug 2025-02-10 19:53:47 -05:00
c10c3a0beb cookies 2025-02-08 08:56:19 -05:00
cc365970a9 Squashed commit of the following:
commit 893b229cc6
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 22:14:08 2025 -0500

    j

commit 06f539aea2
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:55:30 2025 -0500

    fk

commit d0c2960ec9
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:50:53 2025 -0500

    frick

commit 6b1362c166
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:49:40 2025 -0500

    database

commit 8cadc6df4c
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:38:09 2025 -0500

    asdf

commit 1ca6f98684
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:32:50 2025 -0500

    fffff

commit 8bb337a9c3
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:31:13 2025 -0500

    ffff

commit 65aba280c5
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:26:16 2025 -0500

    aa

commit 59ef03a59e
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:24:21 2025 -0500

    asdf

commit f44d5740fc
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:23:32 2025 -0500

    aaa

commit 13c96b1643
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:18:54 2025 -0500

    sdf

commit 949c795fd1
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 21:17:53 2025 -0500

    asdf

commit 8c3cd423fe
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:56:01 2025 -0500

    app2

commit 78eafc739e
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:54:55 2025 -0500

    app

commit dc47eced14
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:43:15 2025 -0500

    asdfasdfasdf

commit e24bcae88c
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:39:44 2025 -0500

    a

commit c894451bfe
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:38:20 2025 -0500

    req

commit 3d09869562
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:33:27 2025 -0500

    wrong number = code dont work lol i love computers

commit 4c93a1271b
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 20:29:39 2025 -0500

    q

commit 1f5361da88
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 18:27:20 2025 -0500

    same as original code now -5 days of my life

commit 511b070cbb
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 13:52:28 2025 -0500

    pricey worky

commit 964fdd641b
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Fri Feb 7 11:37:29 2025 -0500

    prep for pricing service work

commit a78c3bcba3
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Wed Feb 5 21:51:22 2025 -0500

    more stuff yay

commit bd9cfca7a9
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Tue Feb 4 22:30:33 2025 -0500

    GIGA FIXED EVERYTHING OMG

commit 85510a4671
Author: zman <joshua.k.rzemien@gmail.com>
Date:   Tue Feb 4 00:01:34 2025 -0500

    data model change and some new services
2025-02-07 22:20:34 -05:00
46 changed files with 1449 additions and 152 deletions

3
.gitignore vendored
View File

@ -173,3 +173,6 @@ cython_debug/
temp/ temp/
.DS_Store .DS_Store
*.db-journal *.db-journal
cookies/
alembic/versions/*
*.csv

15
Dockerfile Normal file
View File

@ -0,0 +1,15 @@
FROM python:3.13-slim
WORKDIR /app
ENV DATABASE_URL=postgresql://poggers:giga!@192.168.1.41:5432/omegatcgdb
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

119
alembic.ini Normal file
View File

@ -0,0 +1,119 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts
# Use forward slashes (/) also on windows to provide an os agnostic path
script_location = alembic
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = .
# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =
# max length of characters to apply to the "slug" field
# truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; This defaults
# to alembic/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
# version_path_separator = newline
#
# Use os.pathsep. Default configuration used for new projects.
version_path_separator = os
# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = sqlite:///omegacard.db
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples
# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
# hooks = ruff
# ruff.type = exec
# ruff.executable = %(here)s/.venv/bin/ruff
# ruff.options = --fix REVISION_SCRIPT_FILENAME
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

1
alembic/README Normal file
View File

@ -0,0 +1 @@
Generic single-database configuration.

90
alembic/env.py Normal file
View File

@ -0,0 +1,90 @@
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from alembic import context
import sys
import os
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from app.db.models import Base
from app.db.database import DATABASE_URL
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
config.set_main_option('sqlalchemy.url', DATABASE_URL)
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = Base.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection, target_metadata=target_metadata
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

26
alembic/script.py.mako Normal file
View File

@ -0,0 +1,26 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}

View File

@ -0,0 +1,30 @@
"""empty message
Revision ID: f629adc7e597
Revises:
Create Date: 2025-02-07 20:13:32.559672
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'f629adc7e597'
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
pass
# ### end Alembic commands ###

View File

@ -4,10 +4,10 @@ from contextlib import contextmanager
from typing import Generator from typing import Generator
import os import os
from sqlalchemy import inspect from sqlalchemy import inspect
from services.tcgplayer import TCGPlayerService from app.services.tcgplayer import TCGPlayerService
from services.pricing import PricingService from app.services.pricing import PricingService
from services.file import FileService from app.services.file import FileService
from db.models import Price from app.db.models import Price
from datetime import datetime from datetime import datetime

View File

@ -132,7 +132,7 @@ class Card(Base):
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now) date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
class CardManabox(Base): class CardManabox(Base):
__tablename__ = "card_manabox" __tablename__ = "manabox_cards"
product_id = Column(String, ForeignKey("cards.product_id"), primary_key=True) product_id = Column(String, ForeignKey("cards.product_id"), primary_key=True)
name = Column(String) name = Column(String)
@ -147,10 +147,10 @@ class CardManabox(Base):
language = Column(String) language = Column(String)
class CardTCGPlayer(Base): class CardTCGPlayer(Base):
__tablename__ = "card_tcgplayer" __tablename__ = "tcgplayer_cards"
product_id = Column(String, ForeignKey("cards.product_id"), primary_key=True) product_id = Column(String, ForeignKey("cards.product_id"), primary_key=True)
group_id = Column(Integer, ForeignKey("tcgplayer_groups.group_id")) group_id = Column(Integer)
tcgplayer_id = Column(Integer) tcgplayer_id = Column(Integer)
product_line = Column(String) product_line = Column(String)
set_name = Column(String) set_name = Column(String)
@ -164,7 +164,7 @@ class Warehouse(Base):
""" """
container that is associated with a user and contains inventory and stock container that is associated with a user and contains inventory and stock
""" """
__tablename__ = "warehouse" __tablename__ = "warehouses"
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
user_id = Column(String, ForeignKey("users.id")) user_id = Column(String, ForeignKey("users.id"))
@ -175,10 +175,10 @@ class Stock(Base):
""" """
contains products that are listed for sale contains products that are listed for sale
""" """
__tablename__ = "stock" __tablename__ = "stocks"
product_id = Column(String, ForeignKey("products.id"), primary_key=True) product_id = Column(String, ForeignKey("products.id"), primary_key=True)
warehouse_id = Column(String, ForeignKey("warehouse.id"), default="default") warehouse_id = Column(String, ForeignKey("warehouses.id"), default="default")
marketplace_id = Column(String, ForeignKey("marketplaces.id")) marketplace_id = Column(String, ForeignKey("marketplaces.id"))
quantity = Column(Integer) quantity = Column(Integer)
date_created = Column(DateTime, default=datetime.now) date_created = Column(DateTime, default=datetime.now)
@ -190,10 +190,10 @@ class Inventory(Base):
sealed product in breakdown queue, held sealed product, speculatively held singles, etc. sealed product in breakdown queue, held sealed product, speculatively held singles, etc.
inventory can contain products across multiple marketplaces inventory can contain products across multiple marketplaces
""" """
__tablename__ = "inventory" __tablename__ = "inventories"
product_id = Column(String, ForeignKey("products.id"), primary_key=True) product_id = Column(String, ForeignKey("products.id"), primary_key=True)
warehouse_id = Column(String, ForeignKey("warehouse.id"), default="default") warehouse_id = Column(String, ForeignKey("warehouses.id"), default="default")
quantity = Column(Integer) quantity = Column(Integer)
date_created = Column(DateTime, default=datetime.now) date_created = Column(DateTime, default=datetime.now)
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now) date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
@ -243,12 +243,12 @@ class File(Base):
filepath = Column(String) # backup location filepath = Column(String) # backup location
filesize_kb = Column(Float) filesize_kb = Column(Float)
status = Column(String) status = Column(String)
box_id = Column(String, ForeignKey("boxes.product_id"), nullable=True) box_id = Column(String, nullable=True)
date_created = Column(DateTime, default=datetime.now) date_created = Column(DateTime, default=datetime.now)
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now) date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
class Price(Base): class Price(Base):
__tablename__ = "price" __tablename__ = "prices"
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
product_id = Column(String, ForeignKey("products.id")) product_id = Column(String, ForeignKey("products.id"))
@ -271,7 +271,7 @@ class StorageBlock(Base):
return type return type
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
warehouse_id = Column(String, ForeignKey("warehouse.id")) warehouse_id = Column(String, ForeignKey("warehouses.id"))
name = Column(String) name = Column(String)
type = Column(String) # rare or common type = Column(String) # rare or common
date_created = Column(DateTime, default=datetime.now) date_created = Column(DateTime, default=datetime.now)
@ -282,7 +282,7 @@ class ProductBlock(Base):
ProductBlock represents the relationship between a product and a storage block ProductBlock represents the relationship between a product and a storage block
which products are in a block and at what index which products are in a block and at what index
""" """
__tablename__ = "product_block" __tablename__ = "product_blocks"
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
product_id = Column(String, ForeignKey("products.id")) product_id = Column(String, ForeignKey("products.id"))
@ -295,7 +295,7 @@ class OpenBoxCard(Base):
""" """
OpenedBoxCard represents the relationship between an opened box and the cards it contains OpenedBoxCard represents the relationship between an opened box and the cards it contains
""" """
__tablename__ = "open_box_card" __tablename__ = "open_box_cards"
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
open_box_id = Column(String, ForeignKey("open_boxes.id")) open_box_id = Column(String, ForeignKey("open_boxes.id"))
@ -308,7 +308,7 @@ class ProductSale(Base):
""" """
ProductSale represents the relationship between products and sales ProductSale represents the relationship between products and sales
""" """
__tablename__ = "product_sale" __tablename__ = "product_sales"
id = Column(String, primary_key=True) id = Column(String, primary_key=True)
product_id = Column(String, ForeignKey("products.id")) product_id = Column(String, ForeignKey("products.id"))
@ -328,6 +328,73 @@ class TCGPlayerGroups(Base):
modified_on = Column(String) modified_on = Column(String)
category_id = Column(Integer) category_id = Column(Integer)
class Orders(Base):
__tablename__ = 'orders'
id = Column(String, primary_key=True)
order_id = Column(String, unique=True)
buyer_name = Column(String)
recipient_name = Column(String)
recipient_address_one = Column(String)
recipient_address_two = Column(String)
recipient_city = Column(String)
recipient_state = Column(String)
recipient_zip = Column(String)
recipient_country = Column(String)
order_date = Column(String)
status = Column(String)
num_products = Column(Integer)
num_cards = Column(Integer)
product_amount = Column(Float)
shipping_amount = Column(Float)
gross_amount = Column(Float)
fee_amount = Column(Float)
net_amount = Column(Float)
direct_fee_amount = Column(Float)
date_created = Column(DateTime, default=datetime.now)
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
class OrderProducts(Base):
__tablename__ = 'order_products'
id = Column(String, primary_key=True)
order_id = Column(String, ForeignKey('orders.id'))
product_id = Column(String, ForeignKey('products.id'))
quantity = Column(Integer)
unit_price = Column(Float)
class APIPricing(Base):
__tablename__ = 'api_pricing'
id = Column(String, primary_key=True)
product_id = Column(String, ForeignKey('products.id'))
pricing_data = Column(String)
date_created = Column(DateTime, default=datetime.now)
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
class TCGPlayerInventory(Base):
__tablename__ = 'tcgplayer_inventory'
id = Column(String, primary_key=True)
tcgplayer_id = Column(Integer)
product_line = Column(String)
set_name = Column(String)
product_name = Column(String)
title = Column(String)
number = Column(String)
rarity = Column(String)
condition = Column(String)
tcg_market_price = Column(Float)
tcg_direct_low = Column(Float)
tcg_low_price_with_shipping = Column(Float)
tcg_low_price = Column(Float)
total_quantity = Column(Integer)
add_to_quantity = Column(Integer)
tcg_marketplace_price = Column(Float)
photo_url = Column(String)
date_created = Column(DateTime, default=datetime.now)
date_modified = Column(DateTime, default=datetime.now, onupdate=datetime.now)
# enums # enums
class RarityEnum(str, Enum): class RarityEnum(str, Enum):

View File

@ -1,6 +1,6 @@
from contextlib import contextmanager from contextlib import contextmanager
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from exceptions import FailedUploadException from app.exceptions import FailedUploadException
import logging import logging
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)

View File

@ -2,22 +2,27 @@ from typing import Annotated
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from fastapi import Depends, Form from fastapi import Depends, Form
from services.box import BoxService from app.services.box import BoxService
from services.tcgplayer import TCGPlayerService from app.services.tcgplayer import TCGPlayerService
from services.pricing import PricingService from app.services.pricing import PricingService
from services.file import FileService from app.services.file import FileService
from services.product import ProductService from app.services.product import ProductService
from services.inventory import InventoryService from app.services.inventory import InventoryService
from services.task import TaskService from app.services.task import TaskService
from services.storage import StorageService from app.services.storage import StorageService
from db.database import get_db from app.services.tcgplayer_api import TCGPlayerAPIService
from schemas.file import CreateFileRequest from app.db.database import get_db
from schemas.box import CreateBoxRequest, UpdateBoxRequest, CreateOpenBoxRequest from app.schemas.file import CreateFileRequest
from app.schemas.box import CreateBoxRequest, UpdateBoxRequest, CreateOpenBoxRequest
# Common type annotation for database dependency # Common type annotation for database dependency
DB = Annotated[Session, Depends(get_db)] DB = Annotated[Session, Depends(get_db)]
# Base Services (no dependencies besides DB) # Base Services (no dependencies besides DB)
def get_tcgplayer_api_service(db: DB) -> TCGPlayerAPIService:
"""TCGPlayerAPIService with only database dependency"""
return TCGPlayerAPIService(db)
def get_file_service(db: DB) -> FileService: def get_file_service(db: DB) -> FileService:
"""FileService with only database dependency""" """FileService with only database dependency"""
return FileService(db) return FileService(db)
@ -61,10 +66,11 @@ def get_box_service(
def get_task_service( def get_task_service(
db: DB, db: DB,
product_service: Annotated[ProductService, Depends(get_product_service)], product_service: Annotated[ProductService, Depends(get_product_service)],
pricing_service: Annotated[PricingService, Depends(get_pricing_service)] pricing_service: Annotated[PricingService, Depends(get_pricing_service)],
tcgplayer_api_service: Annotated[TCGPlayerAPIService, Depends(get_tcgplayer_api_service)]
) -> TaskService: ) -> TaskService:
"""TaskService depends on ProductService and TCGPlayerService""" """TaskService depends on ProductService and TCGPlayerService"""
return TaskService(db, product_service, pricing_service) return TaskService(db, product_service, pricing_service, tcgplayer_api_service)
# Form data dependencies # Form data dependencies
def get_create_file_metadata( def get_create_file_metadata(

View File

@ -1,13 +1,13 @@
from fastapi import FastAPI from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
import uvicorn import uvicorn
from routes.routes import router from app.routes.routes import router
from db.database import init_db, check_db_connection, get_db from app.db.database import init_db, check_db_connection, get_db
import logging import logging
import sys import sys
# Import your dependency functions # Import your dependency functions
from dependencies import ( from app.dependencies import (
get_task_service, get_task_service,
get_tcgplayer_service, get_tcgplayer_service,
get_pricing_service, get_pricing_service,
@ -15,6 +15,7 @@ from dependencies import (
get_product_service, get_product_service,
get_storage_service, get_storage_service,
get_inventory_service, get_inventory_service,
get_tcgplayer_api_service
) )
logging.basicConfig( logging.basicConfig(
@ -69,7 +70,8 @@ async def startup_event():
tcgplayer_service = get_tcgplayer_service(db, file_service) tcgplayer_service = get_tcgplayer_service(db, file_service)
pricing_service = get_pricing_service(db, file_service, tcgplayer_service) pricing_service = get_pricing_service(db, file_service, tcgplayer_service)
product_service = get_product_service(db, file_service, tcgplayer_service, storage_service) product_service = get_product_service(db, file_service, tcgplayer_service, storage_service)
task_service = get_task_service(db, product_service, pricing_service) tcgplayer_api_service = get_tcgplayer_api_service(db)
task_service = get_task_service(db, product_service, pricing_service, tcgplayer_api_service)
# Start task service # Start task service
await task_service.start() await task_service.start()

View File

@ -1,11 +1,14 @@
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File, BackgroundTasks from fastapi import APIRouter, Depends, HTTPException, UploadFile, File, BackgroundTasks, Request
from fastapi.responses import StreamingResponse from fastapi.responses import StreamingResponse
from typing import Optional, List from typing import Optional, List
from io import BytesIO from io import BytesIO
import logging import logging
from datetime import datetime from datetime import datetime
import os
import json
from pydantic import BaseModel
from schemas.file import ( from app.schemas.file import (
FileSchema, FileSchema,
CreateFileRequest, CreateFileRequest,
CreateFileResponse, CreateFileResponse,
@ -13,7 +16,7 @@ from schemas.file import (
DeleteFileResponse, DeleteFileResponse,
GetFileQueryParams GetFileQueryParams
) )
from schemas.box import ( from app.schemas.box import (
CreateBoxResponse, CreateBoxResponse,
CreateBoxRequest, CreateBoxRequest,
BoxSchema, BoxSchema,
@ -22,11 +25,13 @@ from schemas.box import (
CreateOpenBoxResponse, CreateOpenBoxResponse,
OpenBoxSchema OpenBoxSchema
) )
from services.file import FileService from app.schemas.orders import ProcessOrdersResponse
from services.box import BoxService from app.services.file import FileService
from services.task import TaskService from app.services.box import BoxService
from services.pricing import PricingService from app.services.task import TaskService
from dependencies import ( from app.services.pricing import PricingService
from app.services.tcgplayer_api import TCGPlayerAPIService
from app.dependencies import (
get_file_service, get_file_service,
get_box_service, get_box_service,
get_task_service, get_task_service,
@ -34,7 +39,8 @@ from dependencies import (
get_box_data, get_box_data,
get_box_update_data, get_box_update_data,
get_open_box_data, get_open_box_data,
get_pricing_service get_pricing_service,
get_tcgplayer_api_service
) )
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -230,16 +236,18 @@ async def delete_open_box(
raise HTTPException(status_code=400, detail=str(e) raise HTTPException(status_code=400, detail=str(e)
) )
class InventoryAddRequest(BaseModel):
open_box_ids: List[str]
@router.post("/tcgplayer/inventory/add", response_class=StreamingResponse) @router.post("/tcgplayer/inventory/add", response_class=StreamingResponse)
async def create_inventory_add_file( async def create_inventory_add_file(
request: dict, # Just use a dict instead body: InventoryAddRequest,
pricing_service: PricingService = Depends(get_pricing_service), pricing_service: PricingService = Depends(get_pricing_service),
): ):
"""Create a new inventory add file for download.""" """Create a new inventory add file for download."""
try: try:
# Get IDs directly from the dict content = pricing_service.generate_tcgplayer_inventory_update_file_with_pricing(body.open_box_ids)
open_box_ids = request.get('open_box_ids', [])
content = pricing_service.generate_tcgplayer_inventory_update_file_with_pricing(open_box_ids)
stream = BytesIO(content) stream = BytesIO(content)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
@ -276,3 +284,55 @@ async def create_inventory_update_file(
except Exception as e: except Exception as e:
logger.error(f"Create inventory update file failed: {str(e)}") logger.error(f"Create inventory update file failed: {str(e)}")
raise HTTPException(status_code=400, detail=str(e)) raise HTTPException(status_code=400, detail=str(e))
class CookieUpdate(BaseModel):
cookies: dict
# cookies
@router.post("/cookies", response_model=dict)
async def update_cookies(
cookie_data: CookieUpdate
):
try:
# see if cookie file exists
if not os.path.exists('cookies') or os.path.exists('cookies/tcg_cookies.json'):
logger.info("Cannot find cookies")
# Create cookies directory if it doesn't exist
os.makedirs('cookies', exist_ok=True)
# Save cookies with timestamp
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
cookie_path = 'cookies/tcg_cookies.json'
# Save new cookies
with open(cookie_path, 'w') as f:
json.dump(cookie_data.cookies, f, indent=2)
return {"message": "Cookies updated successfully"}
except Exception as e:
raise HTTPException(
status_code=500,
detail=f"Failed to update cookies: {str(e)}"
)
class TCGPlayerOrderRequest(BaseModel):
order_ids: List[str]
@router.post("/processOrders", response_model=ProcessOrdersResponse)
async def process_orders(
body: TCGPlayerOrderRequest,
tcgplayer_api_service: TCGPlayerAPIService = Depends(get_tcgplayer_api_service),
) -> ProcessOrdersResponse:
"""Process TCGPlayer orders."""
try:
orders = tcgplayer_api_service.process_orders(body.order_ids)
return ProcessOrdersResponse(
status_code=200,
success=True,
orders=orders
)
except Exception as e:
logger.error(f"Process orders failed: {str(e)}")
raise HTTPException(status_code=400, detail=str(e))

View File

@ -1,5 +1,5 @@
from pydantic import BaseModel, Field, ConfigDict from pydantic import BaseModel, Field, ConfigDict
from schemas.base import BaseSchema from app.schemas.base import BaseSchema
from typing import Optional from typing import Optional
from datetime import datetime from datetime import datetime
@ -57,7 +57,7 @@ class CreateOpenBoxRequest(BaseModel):
product_id: str = Field(..., title="Product ID") product_id: str = Field(..., title="Product ID")
file_ids: list[str] = Field(None, title="File IDs") file_ids: list[str] = Field(None, title="File IDs")
num_cards_actual: Optional[int] = Field(None, title="Number of cards actual") num_cards_actual: Optional[int] = Field(None, title="Number of cards actual")
date_opened: Optional [str] = Field(None, title="Date Opened") date_opened: Optional[str] = Field(None, title="Date Opened")
# RESPONSE # RESPONSE
class CreateOpenBoxResponse(BaseModel): class CreateOpenBoxResponse(BaseModel):

9
app/schemas/orders.py Normal file
View File

@ -0,0 +1,9 @@
from pydantic import BaseModel
class OrderSchema(BaseModel):
order_id: str
class ProcessOrdersResponse(BaseModel):
status_code: int
success: bool
orders: list[str]

View File

@ -5,7 +5,7 @@ from sqlalchemy import or_
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
import logging import logging
from db.models import ( from app.db.models import (
Box, Box,
File, File,
StagedFileProduct, StagedFileProduct,
@ -15,9 +15,9 @@ from db.models import (
TCGPlayerGroups, TCGPlayerGroups,
Inventory Inventory
) )
from db.utils import db_transaction from app.db.utils import db_transaction
from schemas.box import CreateBoxRequest, UpdateBoxRequest, CreateOpenBoxRequest from app.schemas.box import CreateBoxRequest, UpdateBoxRequest, CreateOpenBoxRequest
from services.inventory import InventoryService from app.services.inventory import InventoryService
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -45,7 +45,7 @@ class BoxService:
def add_products_to_open_box(self, open_box: OpenBox, product_data: Dict[Product, int]) -> None: def add_products_to_open_box(self, open_box: OpenBox, product_data: Dict[Product, int]) -> None:
"""Add products to an open box.""" """Add products to an open box."""
for product, quantity in product_data.items(): for product, quantity in product_data.items(): # TODO BATCH THIS
open_box_card = OpenBoxCard( open_box_card = OpenBoxCard(
id=str(uuid4()), id=str(uuid4()),
open_box_id=open_box.id, open_box_id=open_box.id,
@ -86,6 +86,8 @@ class BoxService:
type='box', type='box',
product_line='mtg' product_line='mtg'
) )
self.db.add(product)
self.db.flush()
box = Box( box = Box(
product_id=product.id, product_id=product.id,
type=create_box_data.type, type=create_box_data.type,
@ -93,7 +95,6 @@ class BoxService:
sku=create_box_data.sku, sku=create_box_data.sku,
num_cards_expected=create_box_data.num_cards_expected num_cards_expected=create_box_data.num_cards_expected
) )
self.db.add(product)
self.db.add(box) self.db.add(box)
return box, True return box, True

View File

@ -6,15 +6,15 @@ import logging
import os import os
from io import StringIO from io import StringIO
from db.utils import db_transaction from app.db.utils import db_transaction
from db.models import File, StagedFileProduct from app.db.models import File, StagedFileProduct
from schemas.file import CreateFileRequest from app.schemas.file import CreateFileRequest
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class FileConfig: class FileConfig:
"""Configuration constants for file processing""" """Configuration constants for file processing"""
TEMP_DIR = os.path.join(os.getcwd(), 'temp') TEMP_DIR = os.path.join(os.getcwd(), 'app/' + 'temp')
MANABOX_HEADERS = [ MANABOX_HEADERS = [
'Name', 'Set code', 'Set name', 'Collector number', 'Foil', 'Name', 'Set code', 'Set name', 'Collector number', 'Foil',

View File

@ -3,9 +3,9 @@ from typing import Dict
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from sqlalchemy.exc import SQLAlchemyError from sqlalchemy.exc import SQLAlchemyError
from db.models import Product, Inventory from app.db.models import Product, Inventory
from schemas.inventory import UpdateInventoryResponse from app.schemas.inventory import UpdateInventoryResponse
from db.utils import db_transaction from app.db.utils import db_transaction
class InventoryService: class InventoryService:
@ -38,7 +38,8 @@ class InventoryService:
if inventory is None: if inventory is None:
inventory = Inventory( inventory = Inventory(
product_id=product.id, product_id=product.id,
quantity=quantity quantity=quantity,
warehouse_id="0f0d01b1-97ba-4ab2-9082-22062bca9b06" # TODO FIX
) )
self.db.add(inventory) self.db.add(inventory)
else: else:
@ -61,7 +62,7 @@ class InventoryService:
""" """
try: try:
with db_transaction(self.db): with db_transaction(self.db):
for product, quantity in product_data.items(): for product, quantity in product_data.items(): # TODO BATCH THIS
self.add_inventory(product, quantity) self.add_inventory(product, quantity)
return UpdateInventoryResponse(success=True) return UpdateInventoryResponse(success=True)
except SQLAlchemyError: except SQLAlchemyError:

View File

@ -1,11 +1,12 @@
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from db.models import File, CardTCGPlayer, Price from app.db.models import File, CardTCGPlayer, Price, TCGPlayerInventory
from services.util._dataframe import TCGPlayerPricingRow, DataframeUtil from app.services.util._dataframe import TCGPlayerPricingRow, DataframeUtil
from services.file import FileService from app.services.file import FileService
from services.tcgplayer import TCGPlayerService from app.services.tcgplayer import TCGPlayerService
from uuid import uuid4 from uuid import uuid4
from db.utils import db_transaction from app.db.utils import db_transaction
from typing import List, Dict from typing import List, Dict
from decimal import Decimal, ROUND_HALF_UP
import pandas as pd import pandas as pd
import logging import logging
@ -93,6 +94,7 @@ class PricingService:
def cron_load_prices(self, file: File = None): def cron_load_prices(self, file: File = None):
file_content = self.get_pricing_export_content(file) file_content = self.get_pricing_export_content(file)
self.tcgplayer_service.load_tcgplayer_cards(file_content)
self.load_pricing_csv_content_to_db(file_content) self.load_pricing_csv_content_to_db(file_content)
def get_all_prices_for_products(self, product_ids: List[str]) -> Dict[str, Dict[str, float]]: def get_all_prices_for_products(self, product_ids: List[str]) -> Dict[str, Dict[str, float]]:
@ -115,25 +117,52 @@ class PricingService:
def default_pricing_algo(self, row: pd.Series) -> pd.Series: def default_pricing_algo(self, row: pd.Series) -> pd.Series:
"""Default pricing algorithm with complex pricing rules""" """Default pricing algorithm with complex pricing rules"""
tcg_low = row.get('tcg_low_price')
tcg_low_shipping = row.get('tcg_low_price_with_shipping')
if pd.isna(tcg_low) or pd.isna(tcg_low_shipping): # Convert input values to Decimal for precise arithmetic
tcg_low = Decimal(str(row.get('tcg_low_price'))) if not pd.isna(row.get('tcg_low_price')) else None
tcg_low_shipping = Decimal(str(row.get('tcg_low_price_with_shipping'))) if not pd.isna(row.get('tcg_low_price_with_shipping')) else None
tcg_market_price = Decimal(str(row.get('tcg_market_price'))) if not pd.isna(row.get('tcg_market_price')) else None
total_quantity = str(row.get('total_quantity')) if not pd.isna(row.get('total_quantity')) else "0"
added_quantity = str(row.get('add_to_quantity')) if not pd.isna(row.get('add_to_quantity')) else "0"
quantity = int(total_quantity) + int(added_quantity)
if tcg_market_price is None:
logger.warning(f"Missing pricing data for row: {row}") logger.warning(f"Missing pricing data for row: {row}")
row['new_price'] = None row['new_price'] = None
return row return row
# Apply pricing rules # Define precision for rounding
if tcg_low < 0.35: TWO_PLACES = Decimal('0.01')
new_price = 0.35
elif tcg_low < 5 or tcg_low_shipping < 5:
new_price = round(tcg_low * 1.25, 2)
elif tcg_low_shipping > 25:
new_price = round(tcg_low_shipping * 1.025, 2)
else:
new_price = round(tcg_low_shipping * 1.10, 2)
row['new_price'] = new_price # Apply pricing rules
if tcg_market_price < Decimal('1') and tcg_market_price > Decimal('0.25'):
new_price = tcg_market_price * Decimal('1.25')
elif tcg_market_price < Decimal('0.25'):
new_price = Decimal('0.25')
elif tcg_market_price < Decimal('5'):
new_price = tcg_market_price * Decimal('1.08')
elif tcg_market_price < Decimal('10'):
new_price = tcg_market_price * Decimal('1.06')
elif tcg_market_price < Decimal('20'):
new_price = tcg_market_price * Decimal('1.0125')
elif tcg_market_price < Decimal('50'):
new_price = tcg_market_price * Decimal('0.99')
elif tcg_market_price < Decimal('100'):
new_price = tcg_market_price * Decimal('0.98')
else:
new_price = tcg_market_price * Decimal('1.09')
if new_price < Decimal('0.25'):
new_price = Decimal('0.25')
if quantity > 3:
new_price = new_price * Decimal('1.1')
# Ensure exactly 2 decimal places
new_price = new_price.quantize(TWO_PLACES, rounding=ROUND_HALF_UP)
# Convert back to float or string as needed for your dataframe
row['new_price'] = float(new_price)
return row return row
def apply_pricing_algo(self, row: pd.Series, pricing_algo: callable = None) -> pd.Series: def apply_pricing_algo(self, row: pd.Series, pricing_algo: callable = None) -> pd.Series:
@ -187,6 +216,8 @@ class PricingService:
# Set marketplace price # Set marketplace price
df['TCG Marketplace Price'] = df['new_price'] df['TCG Marketplace Price'] = df['new_price']
df['Title'] = ''
column_mapping = { column_mapping = {
'tcgplayer_id': 'TCGplayer Id', 'tcgplayer_id': 'TCGplayer Id',
'product_line': 'Product Line', 'product_line': 'Product Line',
@ -209,6 +240,19 @@ class PricingService:
# Now do your column selection # Now do your column selection
df = df[desired_columns] df = df[desired_columns]
if update_type == 'update':
with db_transaction(self.db):
self.db.query(TCGPlayerInventory).delete()
self.db.flush()
# copy df to modify before inserting
df_copy = df.copy()
df_copy['id'] = df_copy.apply(lambda x: str(uuid4()), axis=1)
# rename columns lowercase no space
df_copy.columns = df_copy.columns.str.lower().str.replace(' ', '_')
for index, row in df_copy.iterrows():
tcgplayer_inventory = TCGPlayerInventory(**row.to_dict())
self.db.add(tcgplayer_inventory)
# remove any rows with no price # remove any rows with no price
#df = df[df['TCG Marketplace Price'] != 0] #df = df[df['TCG Marketplace Price'] != 0]
#df = df[df['TCG Marketplace Price'].notna()] #df = df[df['TCG Marketplace Price'].notna()]

157
app/services/print.py Normal file
View File

@ -0,0 +1,157 @@
from brother_ql.conversion import convert
from brother_ql.backends.helpers import send
from brother_ql.raster import BrotherQLRaster
from PIL import Image, ImageDraw, ImageFont
import platform
import pandas as pd
from time import sleep
import pdf2image
import io
# Printer settings
printer_model = "QL-1100"
backend = 'pyusb'
printer = 'usb://0x04f9:0x20a7'
def convert_pdf_to_image(pdf_path):
"""Converts a PDF to PIL Image"""
try:
# Convert PDF to image
images = pdf2image.convert_from_path(pdf_path)
if images:
return images[0] # Return first page
return None
except Exception as e:
print(f"Error converting PDF: {str(e)}")
return None
def create_address_label(input_data, font_size=30, is_pdf=False):
"""Creates and returns the label image without printing"""
if is_pdf:
if isinstance(input_data, str): # If path is provided
return convert_pdf_to_image(input_data)
else: # If PIL Image is provided
return input_data
# Regular text-based label creation
label_width = 991
label_height = 306
image = Image.new('L', (label_width, label_height), 'white')
draw = ImageDraw.Draw(image)
# Font selection based on OS
if platform.system() == 'Windows':
font = ImageFont.truetype("C:\\Windows\\Fonts\\arial.ttf", size=font_size)
elif platform.system() == 'Darwin':
font = ImageFont.truetype("/Library/Fonts/Arial.ttf", size=font_size)
elif platform.system() == 'Linux':
font = ImageFont.truetype("/usr/share/fonts/truetype/msttcorefonts/arial.ttf", size=font_size)
margin = 20
lines = input_data.split('\n')
line_height = font_size + 5
total_height = line_height * len(lines)
start_y = (label_height - total_height) // 2
for i, line in enumerate(lines):
y = start_y + (i * line_height)
draw.text((margin, y), line, font=font, fill='black')
return image
def preview_label(input_data, font_size=30, is_pdf=False):
"""Creates and displays the label preview"""
image = create_address_label(input_data, font_size, is_pdf)
if image:
image.show()
def print_address_label(input_data, font_size=30, is_pdf=False, label_size='29x90'):
"""Prints the label with support for both text and PDF inputs"""
try:
image = create_address_label(input_data, font_size, is_pdf)
if not image:
raise Exception("Failed to create label image")
if label_size == '4x6':
target_width = 1164
target_height = 1660
image = image.resize((target_width, target_height), Image.LANCZOS)
qlr = BrotherQLRaster(printer_model)
qlr.exception_on_warning = True
print("Converting image to printer instructions...")
instructions = convert(
qlr=qlr,
images=[image],
label='29x90' if label_size == '29x90' else '102x152',
threshold=70.0,
dither=False,
compress=False,
red=False,
dpi_600=False,
hq=True,
cut=False
)
print("Sending to printer...")
send(
instructions=instructions,
printer_identifier=printer,
backend_identifier=backend,
blocking=True
)
print("Print job sent successfully")
except Exception as e:
print(f"Error during printing: {str(e)}")
def process_pirate_ship_pdf(pdf_path, preview=False):
"""Process and print a Pirate Ship PDF shipping label"""
if preview:
preview_label(pdf_path, is_pdf=True)
else:
print_address_label(pdf_path, is_pdf=True, label_size='4x6')
def process_tcg_shipping_export(file_path, require_input=False, font_size=60, preview=False):
# Load the CSV file, all columns are strings
df = pd.read_csv(file_path, dtype=str)
print(df.dtypes)
for i, row in df.iterrows():
line1 = str(row['FirstName']) + ' ' + str(row['LastName'])
line2 = str(row['Address1'])
if not pd.isna(row['Address2']):
line2 += ' ' + str(row['Address2'])
line3 = str(row['City']) + ', ' + str(row['State']) + ' ' + str(row['PostalCode'])
address = f"{line1}\n{line2}\n{line3}"
if preview:
preview_label(address, font_size=font_size)
else:
print_address_label(address, font_size=font_size)
if require_input:
input("Press Enter to continue...")
else:
sleep(1)
if __name__ == "__main__":
# Choose which type to process
label_type = input("Enter label type (1 for regular, 2 for TCG, 3 for Pirate Ship): ")
if label_type == "1":
address = input("Enter the address to print: ")
preview_label(address, font_size=60)
user_input = input("Press 'p' to print the label or any other key to cancel: ")
if user_input.lower() == 'p':
print_address_label(address, font_size=60)
elif label_type == "2":
shipping_export_file = input("Enter the path to the TCG Player shipping export CSV file: ")
process_tcg_shipping_export(shipping_export_file, font_size=60, preview=False)
elif label_type == "3":
pirate_ship_pdf = input("Enter the path to the Pirate Ship PDF file: ")
process_pirate_ship_pdf(pirate_ship_pdf, preview=True)
user_input = input("Press 'p' to print the label or any other key to cancel: ")
if user_input.lower() == 'p':
process_pirate_ship_pdf(pirate_ship_pdf, preview=False)

View File

@ -3,12 +3,12 @@ from uuid import uuid4
from pandas import DataFrame from pandas import DataFrame
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from db.utils import db_transaction from app.db.utils import db_transaction
from db.models import CardManabox, CardTCGPlayer, StagedFileProduct, TCGPlayerGroups from app.db.models import CardManabox, CardTCGPlayer, StagedFileProduct, TCGPlayerGroups
from services.util._dataframe import ManaboxRow, DataframeUtil from app.services.util._dataframe import ManaboxRow, DataframeUtil
from services.file import FileService from app.services.file import FileService
from services.tcgplayer import TCGPlayerService from app.services.tcgplayer import TCGPlayerService
from services.storage import StorageService from app.services.storage import StorageService
logger = getLogger(__name__) logger = getLogger(__name__)

View File

@ -2,8 +2,8 @@ from uuid import uuid4
from typing import List, TypedDict from typing import List, TypedDict
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from db.utils import db_transaction from app.db.utils import db_transaction
from db.models import ( from app.db.models import (
Warehouse, Warehouse,
User, User,
StagedFileProduct, StagedFileProduct,

View File

@ -2,19 +2,21 @@ from apscheduler.schedulers.background import BackgroundScheduler
import logging import logging
from typing import Dict, Callable from typing import Dict, Callable
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from services.product import ProductService from app.services.product import ProductService
from db.models import File from app.db.models import File
from services.pricing import PricingService from app.services.pricing import PricingService
from app.services.tcgplayer_api import TCGPlayerAPIService
class TaskService: class TaskService:
def __init__(self, db: Session, product_service: ProductService, pricing_service: PricingService): def __init__(self, db: Session, product_service: ProductService, pricing_service: PricingService, tcgplayer_api_service: TCGPlayerAPIService):
self.scheduler = BackgroundScheduler() self.scheduler = BackgroundScheduler()
self.logger = logging.getLogger(__name__) self.logger = logging.getLogger(__name__)
self.tasks: Dict[str, Callable] = {} self.tasks: Dict[str, Callable] = {}
self.db = db self.db = db
self.product_service = product_service self.product_service = product_service
self.pricing_service = pricing_service self.pricing_service = pricing_service
self.tcgplayer_api_service = tcgplayer_api_service
async def start(self): async def start(self):
self.scheduler.start() self.scheduler.start()
@ -23,7 +25,9 @@ class TaskService:
# self.pricing_service.generate_tcgplayer_inventory_update_file_with_pricing(['e20cc342-23cb-4593-89cb-56a0cb3ed3f3']) # self.pricing_service.generate_tcgplayer_inventory_update_file_with_pricing(['e20cc342-23cb-4593-89cb-56a0cb3ed3f3'])
def register_scheduled_tasks(self): def register_scheduled_tasks(self):
self.scheduler.add_job(self.hourly_pricing, 'cron', minute='0') self.scheduler.add_job(self.hourly_pricing, 'cron', minute='36')
# every 5 hours on the 24th minute
#self.scheduler.add_job(self.inventory_pricing, 'cron', hour='*', minute='44')
self.logger.info("Scheduled tasks registered.") self.logger.info("Scheduled tasks registered.")
def hourly_pricing(self): def hourly_pricing(self):
@ -31,6 +35,9 @@ class TaskService:
self.pricing_service.cron_load_prices() self.pricing_service.cron_load_prices()
self.logger.info("Finished hourly pricing task") self.logger.info("Finished hourly pricing task")
def inventory_pricing(self):
self.tcgplayer_api_service.cron_tcgplayer_api_pricing()
async def process_manabox_file(self, file: File): async def process_manabox_file(self, file: File):
self.logger.info("Processing ManaBox file") self.logger.info("Processing ManaBox file")
self.product_service.bg_process_manabox_file(file.id) self.product_service.bg_process_manabox_file(file.id)

View File

@ -1,10 +1,10 @@
from db.models import TCGPlayerGroups, CardTCGPlayer, Product, Card, File, Inventory, OpenBox, OpenBoxCard from app.db.models import TCGPlayerGroups, CardTCGPlayer, Product, Card, File, Inventory, OpenBox, OpenBoxCard
import requests import requests
from services.util._dataframe import TCGPlayerPricingRow, DataframeUtil, ManaboxRow from app.services.util._dataframe import TCGPlayerPricingRow, DataframeUtil, ManaboxRow
from services.file import FileService from app.services.file import FileService
from services.inventory import InventoryService from app.services.inventory import InventoryService
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from db.utils import db_transaction from app.db.utils import db_transaction
from uuid import uuid4 as uuid from uuid import uuid4 as uuid
import browser_cookie3 import browser_cookie3
import webbrowser import webbrowser
@ -19,7 +19,10 @@ import time
from typing import List, Dict, Optional from typing import List, Dict, Optional
import pandas as pd import pandas as pd
from sqlalchemy.exc import SQLAlchemyError from sqlalchemy.exc import SQLAlchemyError
from schemas.file import CreateFileRequest from app.schemas.file import CreateFileRequest
import os
from app.services.util._docker import DockerUtil
from sqlalchemy import func
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -52,6 +55,7 @@ class TCGPlayerService:
self.previous_request_time = None self.previous_request_time = None
self.df_util = DataframeUtil() self.df_util = DataframeUtil()
self.file_service = file_service self.file_service = file_service
self.docker_util = DockerUtil()
def _insert_groups(self, groups): def _insert_groups(self, groups):
for group in groups: for group in groups:
@ -98,6 +102,15 @@ class TCGPlayerService:
with db_transaction(self.db): with db_transaction(self.db):
self._insert_groups(groups) self._insert_groups(groups)
def get_cookies_from_file(self) -> Dict:
# check if cookies file exists
if not os.path.exists('cookies/tcg_cookies.json'):
raise ValueError("Cookies file not found")
with open('cookies/tcg_cookies.json', 'r') as f:
logger.debug("Loading cookies from file")
cookies = json.load(f)
return cookies
def _get_browser_cookies(self) -> Optional[Dict]: def _get_browser_cookies(self) -> Optional[Dict]:
"""Retrieve cookies from the specified browser""" """Retrieve cookies from the specified browser"""
try: try:
@ -111,18 +124,26 @@ class TCGPlayerService:
def _send_request(self, url: str, method: str, data=None, except_302=False) -> requests.Response: def _send_request(self, url: str, method: str, data=None, except_302=False) -> requests.Response:
"""Send a request with the specified cookies""" """Send a request with the specified cookies"""
# if previous request was made less than 10 seconds ago, wait until current time is 10 seconds after previous request # Rate limiting logic
if self.previous_request_time: if self.previous_request_time:
time_diff = (datetime.now() - self.previous_request_time).total_seconds() time_diff = (datetime.now() - self.previous_request_time).total_seconds()
if time_diff < 10: if time_diff < 10:
logger.info(f"Waiting 10 seconds before next request...") logger.info(f"Waiting 10 seconds before next request...")
time.sleep(10 - time_diff) time.sleep(10 - time_diff)
headers = self._set_headers(method) headers = self._set_headers(method)
# Move cookie initialization outside and make it more explicit
if not self.cookies: if not self.cookies:
if self.docker_util.is_in_docker():
logger.debug("Running in Docker - using cookies from file")
self.cookies = self.get_cookies_from_file()
else:
logger.debug("Not in Docker - using browser cookies")
self.cookies = self._get_browser_cookies() self.cookies = self._get_browser_cookies()
if not self.cookies: if not self.cookies:
raise ValueError("Failed to retrieve browser cookies") raise ValueError("Failed to retrieve cookies")
try: try:
#logger.info(f"debug: request url {url}, method {method}, data {data}") #logger.info(f"debug: request url {url}, method {method}, data {data}")
@ -459,7 +480,7 @@ class TCGPlayerService:
def get_pricing_export_for_all_products(self) -> File: def get_pricing_export_for_all_products(self) -> File:
""" """
""" """
DEBUG = True DEBUG = False
if DEBUG: if DEBUG:
logger.debug("DEBUG: Using existing pricing export file") logger.debug("DEBUG: Using existing pricing export file")
file = self.db.query(File).filter(File.type == 'tcgplayer_pricing_export').first() file = self.db.query(File).filter(File.type == 'tcgplayer_pricing_export').first()
@ -478,34 +499,49 @@ class TCGPlayerService:
except SQLAlchemyError as e: except SQLAlchemyError as e:
raise RuntimeError(f"Failed to retrieve group IDs: {str(e)}") raise RuntimeError(f"Failed to retrieve group IDs: {str(e)}")
def load_tcgplayer_cards(self) -> File: def load_tcgplayer_cards(self, file_content):
try: try:
# Get pricing export
export_csv_file = self.get_pricing_export_for_all_products()
export_csv = self.file_service.get_file_content(export_csv_file.id)
# load to card tcgplayer # load to card tcgplayer
self.load_export_csv_to_card_tcgplayer(export_csv, export_csv_file.id) self.load_export_csv_to_card_tcgplayer(file_content)
return export_csv_file
except Exception as e: except Exception as e:
logger.error(f"Failed to load prices: {e}") logger.error(f"Failed to load prices: {e}")
raise raise
def open_box_cards_to_tcgplayer_inventory_df(self, open_box_ids: List[str]) -> pd.DataFrame: def open_box_cards_to_tcgplayer_inventory_df(self, open_box_ids: List[str]) -> pd.DataFrame:
tcgcards = (self.db.query(OpenBoxCard, CardTCGPlayer) # Using sqlalchemy to group and sum quantities for duplicate TCGplayer IDs
tcgcards = (self.db.query(
CardTCGPlayer.product_id,
CardTCGPlayer.tcgplayer_id,
CardTCGPlayer.product_line,
CardTCGPlayer.set_name,
CardTCGPlayer.product_name,
CardTCGPlayer.title,
CardTCGPlayer.number,
CardTCGPlayer.rarity,
CardTCGPlayer.condition,
func.sum(OpenBoxCard.quantity).label('quantity')
)
.filter(OpenBoxCard.open_box_id.in_(open_box_ids)) .filter(OpenBoxCard.open_box_id.in_(open_box_ids))
.join(CardTCGPlayer, OpenBoxCard.card_id == CardTCGPlayer.product_id) .join(CardTCGPlayer, OpenBoxCard.card_id == CardTCGPlayer.product_id)
.group_by(
CardTCGPlayer.tcgplayer_id,
CardTCGPlayer.product_id,
CardTCGPlayer.product_line,
CardTCGPlayer.set_name,
CardTCGPlayer.product_name,
CardTCGPlayer.title,
CardTCGPlayer.number,
CardTCGPlayer.rarity,
CardTCGPlayer.condition
)
.all()) .all())
if not tcgcards: if not tcgcards:
return None return None
# Create dataframe # Create dataframe directly from the query results
df = pd.DataFrame([(tcg.product_id, tcg.tcgplayer_id, tcg.product_line, tcg.set_name, tcg.product_name, df = pd.DataFrame(tcgcards,
tcg.title, tcg.number, tcg.rarity, tcg.condition, obc.quantity)
for obc, tcg in tcgcards],
columns=['product_id', 'tcgplayer_id', 'product_line', 'set_name', 'product_name', columns=['product_id', 'tcgplayer_id', 'product_line', 'set_name', 'product_name',
'title', 'number', 'rarity', 'condition', 'quantity']) 'title', 'number', 'rarity', 'condition', 'quantity'])

View File

@ -0,0 +1,322 @@
from dataclasses import dataclass
import logging
from app.db.models import Orders, OrderProducts, CardTCGPlayer, CardManabox, APIPricing, TCGPlayerInventory
from app.services.util._requests import RequestsUtil
from app.services.util._docker import DockerUtil
from app.db.utils import db_transaction
from sqlalchemy.orm import Session
from datetime import datetime
from uuid import uuid4 as uuid
import json
logger = logging.getLogger(__name__)
@dataclass
class TCGPlayerAPIConfig:
"""Configuration for TCGPlayer API"""
ORDER_BASE_URL: str = "https://order-management-api.tcgplayer.com/orders"
API_VERSION: str = "?api-version=2.0"
class TCGPlayerAPIService:
def __init__(self, db: Session):
self.db = db
self.docker_util = DockerUtil()
self.requests_util = RequestsUtil()
self.is_in_docker = self.docker_util.is_in_docker()
self.config = TCGPlayerAPIConfig()
self.cookies = self.get_cookies()
self.session = None
def get_cookies(self) -> dict:
if self.is_in_docker:
return self.requests_util.get_tcgplayer_cookies_from_file()
else:
return self.requests_util.get_tcgplayer_browser_cookies()
def get_order(self, order_id: str) -> dict:
url = f"{self.config.ORDER_BASE_URL}/{order_id}{self.config.API_VERSION}"
response = self.requests_util.send_request(url, method='GET', cookies=self.cookies)
if response:
return response.json()
return None
def get_product_ids_from_sku(self, sku_ids: list[str]) -> dict:
"""Get product IDs from TCGPlayer SKU IDs"""
# convert SKU IDs to integers
sku_ids = [int(sku_id) for sku_id in sku_ids]
tcg_cards = self.db.query(CardTCGPlayer).filter(CardTCGPlayer.tcgplayer_id.in_(sku_ids)).all()
return {str(card.tcgplayer_id): card.product_id for card in tcg_cards}
def save_order(self, order: dict):
# check if order exists by order number
order_number = order['orderNumber']
existing_order = self.db.query(Orders).filter(Orders.order_id == order_number).first()
if existing_order:
logger.info(f"Order {order_number} already exists in database")
return existing_order
transaction = order['transaction']
shipping = order['shippingAddress']
products = order['products']
with db_transaction(self.db):
db_order = Orders(
id = str(uuid()),
order_id=order_number,
buyer_name=order['buyerName'],
recipient_name=shipping['recipientName'],
recipient_address_one=shipping['addressOne'],
recipient_address_two=shipping['addressTwo'] if 'addressTwo' in shipping else '',
recipient_city=shipping['city'],
recipient_state=shipping['territory'],
recipient_zip=shipping['postalCode'],
recipient_country=shipping['country'],
order_date=order['createdAt'],
status=order['status'],
num_products=len(products),
num_cards=sum([product['quantity'] for product in products]),
product_amount=transaction['productAmount'],
shipping_amount=transaction['shippingAmount'],
gross_amount=transaction['grossAmount'],
fee_amount=transaction['feeAmount'],
net_amount=transaction['netAmount'],
direct_fee_amount=transaction['directFeeAmount']
)
self.db.add(db_order)
self.db.flush()
product_ids = [product['skuId'] for product in products]
sku_to_product_id_mapping = self.get_product_ids_from_sku(product_ids)
order_products = []
for product in products:
product_id = sku_to_product_id_mapping.get(product['skuId'])
if product_id:
order_products.append(
OrderProducts(
id=str(uuid()),
order_id=db_order.id,
product_id=product_id,
quantity=product['quantity'],
unit_price=product['unitPrice']
)
)
self.db.add_all(order_products)
return db_order
def process_orders(self, orders: list[str]):
processed_orders = []
for order_id in orders:
order = self.get_order(order_id)
if order:
self.save_order(order)
processed_orders.append(order_id)
return processed_orders
def get_scryfall_data(self, scryfall_id: str):
url = f"https://api.scryfall.com/cards/{scryfall_id}?format=json"
response = self.requests_util.bare_request(url, method='GET')
return response
def get_tcgplayer_pricing_data(self, tcgplayer_id: str):
if not self.session:
self.session = self.requests_util.get_session()
response = self.session.get("https://tcgplayer.com")
headers = {
'accept': 'application/json, text/plain, */*',
'accept-language': 'en-US,en;q=0.8',
'priority': 'u=1, i',
'sec-ch-ua': '"Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-site',
'sec-gpc': '1',
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0 Safari/537.36'
}
url = f"https://mp-search-api.tcgplayer.com/v2/product/{tcgplayer_id}/details?mpfev=3279"
#self.requests_util.rate_limit()
response = self.session.get(url, headers=headers)
self.requests_util.previous_request_time = datetime.now()
return response
# pricing
def get_tcgplayer_pricing_data_for_product(self, product_id: str):
# get tcgplayer pricing data for a single card by product id
# product_id to manabox card
manabox_card = self.db.query(CardManabox).filter(CardManabox.product_id == product_id).first()
tcgplayer_card = self.db.query(CardTCGPlayer).filter(CardTCGPlayer.product_id == product_id).first()
if not manabox_card or not tcgplayer_card:
logger.warning(f"Card with product id {product_id} missing in either Manabox or TCGPlayer")
return None
mbfoil = manabox_card.foil
if str.lower(mbfoil) == 'foil':
logger.warning(f"Card with product id {product_id} is foil, skipping")
return None
# get scryfall id, tcgplayer id, and tcgplayer sku
scryfall_id = manabox_card.scryfall_id
tcgplayer_sku = tcgplayer_card.tcgplayer_id
tcgplayer_id = self.get_scryfall_data(scryfall_id).json().get('tcgplayer_id')
tcgplayer_pricing = self.get_tcgplayer_pricing_data(tcgplayer_id)
if not tcgplayer_pricing:
logger.warning(f"TCGPlayer pricing data not found for product id {product_id}")
return None
else:
logger.info(f"TCGPlayer pricing data found for product id {product_id}")
return tcgplayer_pricing.json()
def save_tcgplayer_pricing_data(self, product_id: str, pricing_data: dict):
# convert to json
pricing_data_json = json.dumps(pricing_data)
with db_transaction(self.db):
pricing_record = APIPricing(
id=str(uuid()),
product_id=product_id,
pricing_data=str(pricing_data_json)
)
self.db.add(pricing_record)
def cron_tcgplayer_api_pricing(self):
# Join both tables but retrieve both objects
results = self.db.query(TCGPlayerInventory, CardTCGPlayer).join(
CardTCGPlayer,
TCGPlayerInventory.tcgplayer_id == CardTCGPlayer.tcgplayer_id
).all()
for inventory, card in results:
# Now use card.product_id (from CardTCGPlayer)
pricing_data = self.get_tcgplayer_pricing_data_for_product(card.product_id)
if pricing_data:
self.save_tcgplayer_pricing_data(card.product_id, pricing_data)
# this one contains nearly everything, use it first
# what does score mean? - totally ignore score, it seems related to price and changes based on what is on the page. probably some psy op shit to get you to buy expensive stuff, not useful for us
# can i get volatility from here?
# no historical data here
"""
curl 'https://mp-search-api.tcgplayer.com/v2/product/615745/details?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; SearchSortSettings=M=1&ProductSortOption=MinPrice&ProductSortDesc=True&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; tcg-segment-session=1740595460137%257C1740595481177' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'
"""
# get volatility also
"""
curl 'https://mpgateway.tcgplayer.com/v1/pricepoints/marketprice/skus/8395586/volatility?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; SearchCriteria=M=1&WantVerifiedSellers=False&WantDirect=False&WantSellersInCart=False; SearchSortSettings=M=1&ProductSortOption=Sales&ProductSortDesc=False&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg-segment-session=1740597680921%257C1740598418227' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'
"""
# handle historical data later
# detailed range quarter - detailed pricing info for the last quarter. seems simple
"""
"""
# listings - lots of stuff here
"""
QUANTITY OVERVIEW
curl 'https://mp-search-api.tcgplayer.com/v1/product/615745/listings?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-H 'content-type: application/json' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; SearchSortSettings=M=1&ProductSortOption=MinPrice&ProductSortDesc=True&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; tcg-segment-session=1740595460137%257C1740595481430' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36' \
--data-raw '{"filters":{"term":{"sellerStatus":"Live","channelId":0,"language":["English"],"direct-seller":true,"directProduct":true,"listingType":"standard"},"range":{"quantity":{"gte":1},"direct-inventory":{"gte":1}},"exclude":{"channelExclusion":0,"listingType":"custom"}},"from":0,"size":1,"context":{"shippingCountry":"US","cart":{}},"sort":{"field":"price+shipping","order":"asc"}}'
AGGREGATION AND SOME SPECIFIC DATA IDK THIS MIGHT BE A GOOD ONE
curl 'https://mp-search-api.tcgplayer.com/v1/product/615745/listings?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-H 'content-type: application/json' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; SearchSortSettings=M=1&ProductSortOption=MinPrice&ProductSortDesc=True&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; tcg-segment-session=1740595460137%257C1740595481430' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36' \
--data-raw '{"filters":{"term":{"sellerStatus":"Live","channelId":0,"language":["English"]},"range":{"quantity":{"gte":1}},"exclude":{"channelExclusion":0}},"from":0,"size":10,"sort":{"field":"price+shipping","order":"asc"},"context":{"shippingCountry":"US","cart":{}},"aggregations":["listingType"]}'
AGGREGATION OF RANDOM SHIT IDK
curl 'https://mp-search-api.tcgplayer.com/v1/product/615745/listings?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-H 'content-type: application/json' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; SearchSortSettings=M=1&ProductSortOption=MinPrice&ProductSortDesc=True&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; tcg-segment-session=1740595460137%257C1740595481430' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36' \
--data-raw '{"filters":{"term":{"condition":["Near Mint","Lightly Played","Moderately Played","Heavily Played","Damaged"],"printing":["Foil"],"language":["English"],"sellerStatus":"Live"},"range":{"quantity":{"gte":1}},"exclude":{"channelExclusion":0}},"context":{"shippingCountry":"US","cart":{}},"aggregations":["seller-key"],"size":0}'
VOLATILITY
curl 'https://mpgateway.tcgplayer.com/v1/pricepoints/marketprice/skus/8547894/volatility?mpfev=3279' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; spDisabledUIFeatures=orders; SellerProximity=ZipCode=&MaxSellerDistance=1000&IsActive=false; tcg-uuid=613192dc-ecf6-481a-bec1-afdee8686db7; LastSeller=e576ed4c; __RequestVerificationToken=VFv72VLK6McJVzthg8O-41p7BNkdoW2jQAlDAu-ylO39qfzCddRi2-7bWiH4qloc8Vo_ZftOAAa5OhXL3OByFHIdlwY1; TCGAuthTicket_Production=270B0566400C905C51DEAD644E3CDBD634ECBCCC796B1F77717F461EE104FCE101CFAD2A8458319330A0931018A99214D4EA5601E7551E25E2069ACA550BB71775C0A04F30724E2C4E262CB167EAC2C2EB05D15F9EA08363FC6455B94654F1F110CF079E24201C3B8CEF26762423D8CAA71DDF7B; ASP.NET_SessionId=5ycv15jf0mon3l5adodmkog5; StoreSaveForLater_PRODUCTION=SFLK=a167bf88521f4d0fbeb7497a7ed74629&Ignore=false; TCG_VisitorKey=81fe992f-9a12-4926-a417-7815c4f94edd; setting=CD=US&M=1; SearchSortSettings=M=1&ProductSortOption=MinPrice&ProductSortDesc=True&PriceSortOption=Shipping&ProductResultDisplay=grid; tcg_analytics_previousPageData=%7B%22title%22%3A%22Seller%20Feedback%22%2C%22href%22%3A%22https%3A%2F%2Fshop.tcgplayer.com%2Fsellerfeedback%2Fbe27fef9%22%7D; fileDownloadToken=1740499419709; StoreCart_PRODUCTION=CK=b4f8aff616974a12a6b2811129b81ee2&Ignore=false; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; tcg-segment-session=1740595460137%257C1740595481430' \
-H 'origin: https://www.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://www.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'
"""

View File

@ -1,6 +1,6 @@
import pandas as pd import pandas as pd
from io import StringIO from io import StringIO
from db.models import File from app.db.models import File
class ManaboxRow: class ManaboxRow:

View File

@ -0,0 +1,49 @@
import os
import logging
logger = logging.getLogger(__name__)
class DockerUtil:
def __init__(self):
pass
def is_in_docker(self) -> bool:
"""Check if we're running inside a Docker container using multiple methods"""
# Method 1: Check cgroup
try:
with open('/proc/1/cgroup', 'r') as f:
content = f.read().lower()
if any(container_id in content for container_id in ['docker', 'containerd', 'kubepods']):
logger.debug("Docker detected via cgroup")
return True
except Exception as e:
logger.debug(f"Could not read cgroup file: {e}")
# Method 2: Check /.dockerenv file
if os.path.exists('/.dockerenv'):
logger.debug("Docker detected via /.dockerenv file")
return True
# Method 3: Check environment variables
docker_env = any(os.environ.get(var, False) for var in [
'DOCKER_CONTAINER',
'IN_DOCKER',
'KUBERNETES_SERVICE_HOST', # For k8s
'DOCKER_HOST'
])
if docker_env:
logger.debug("Docker detected via environment variables")
return True
# Method 4: Check container runtime
try:
with open('/proc/self/mountinfo', 'r') as f:
content = f.read().lower()
if any(rt in content for rt in ['docker', 'containerd', 'kubernetes']):
logger.debug("Docker detected via mountinfo")
return True
except Exception as e:
logger.debug(f"Could not read mountinfo: {e}")
logger.debug("No Docker environment detected")
return False

View File

@ -0,0 +1,191 @@
from typing import Dict, Optional
from app.services.util._docker import DockerUtil
from enum import Enum
import browser_cookie3
import os
import json
import requests
import time
from datetime import datetime
import logging
logger = logging.getLogger(__name__)
class Browser(Enum):
"""Supported browser types for cookie extraction"""
BRAVE = "brave"
CHROME = "chrome"
FIREFOX = "firefox"
class Method(Enum):
"""Supported HTTP methods"""
GET = "GET"
POST = "POST"
class TCGPlayerEndpoints(Enum):
"""Supported TCGPlayer API endpoints"""
ORDERS = "https://order-management-api.tcgplayer.com/orders"
class Headers:
ACCEPT = 'application/json, text/plain, */*'
ACCEPT_ENCODING = 'gzip, deflate, br, zstd'
ACCEPT_LANGUAGE = 'en-US,en;q=0.8'
PRIORITY = 'u=1, i'
SECCHUA = '"Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"'
SECCHUA_MOBILE = '?0'
SECCHUA_PLATFORM = '"macOS"'
SEC_FETCH_DEST = 'empty'
SEC_FETCH_MODE = 'cors'
SEC_FETCH_SITE = 'same-site'
SEC_GPC = '1'
USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'
SELLER_ORIGIN = 'https://sellerportal.tcgplayer.com'
SELLER_REFERER = 'https://sellerportal.tcgplayer.com/'
class RequestHeaders:
BASE_HEADERS = {
'accept': Headers.ACCEPT,
'accept-encoding': Headers.ACCEPT_ENCODING,
'accept-language': Headers.ACCEPT_LANGUAGE,
'priority': Headers.PRIORITY,
'sec-ch-ua': Headers.SECCHUA,
'sec-ch-ua-mobile': Headers.SECCHUA_MOBILE,
'sec-ch-ua-platform': Headers.SECCHUA_PLATFORM,
'sec-fetch-dest': Headers.SEC_FETCH_DEST,
'sec-fetch-mode': Headers.SEC_FETCH_MODE,
'sec-fetch-site': Headers.SEC_FETCH_SITE,
'sec-gpc': Headers.SEC_GPC,
'user-agent': Headers.USER_AGENT
}
SELLER_HEADERS = {
'origin': Headers.SELLER_ORIGIN,
'referer': Headers.SELLER_REFERER
}
class URLHeaders:
# combine base and seller headers
ORDER_HEADERS = {**RequestHeaders.BASE_HEADERS, **RequestHeaders.SELLER_HEADERS}
class RequestsUtil:
def __init__(self, browser_type: Browser = Browser.BRAVE):
self.browser_type = browser_type
self.docker_util = DockerUtil()
self.previous_request_time = datetime.now()
def get_session(self, cookies: Dict = None) -> requests.Session:
"""Create a session with the specified cookies"""
session = requests.Session()
if cookies:
session.cookies.update(cookies)
return session
def bare_request(self, url: str, method: str, cookies: dict = None, data=None) -> requests.Response:
"""Send a request without any additional processing"""
try:
response = requests.request(method, url, cookies=cookies, data=data)
response.raise_for_status()
return response
except requests.RequestException as e:
logger.error(f"Request failed: {str(e)}")
return None
def get_tcgplayer_cookies_from_file(self) -> Dict:
# check if cookies file exists
if not os.path.exists('cookies/tcg_cookies.json'):
raise ValueError("Cookies file not found")
with open('cookies/tcg_cookies.json', 'r') as f:
logger.debug("Loading cookies from file")
cookies = json.load(f)
return cookies
def get_tcgplayer_browser_cookies(self) -> Optional[Dict]:
"""Retrieve cookies from the specified browser"""
try:
cookie_getter = getattr(browser_cookie3, self.browser_type.value, None)
if not cookie_getter:
raise ValueError(f"Unsupported browser type: {self.browser_type.value}")
return cookie_getter()
except Exception as e:
logger.error(f"Failed to get browser cookies: {str(e)}")
return None
def rate_limit(self, time_between_requests: int = 10):
"""Rate limit requests by waiting for a specified time between requests"""
time_diff = (datetime.now() - self.previous_request_time).total_seconds()
if time_diff < time_between_requests:
# logger.info(f"Waiting {time_between_requests - time_diff} seconds before next request...")
time.sleep(time_between_requests - time_diff)
def send_request(self, url: str, method: str, cookies: dict, data=None) -> requests.Response:
"""Send a request with the specified cookies"""
headers = self.set_headers(url)
if not headers:
raise ValueError("Headers not set")
try:
self.rate_limit()
response = requests.request(method, url, headers=headers, cookies=cookies, data=data)
response.raise_for_status()
self.previous_request_time = datetime.now()
return response
except requests.RequestException as e:
logger.error(f"Request failed: {str(e)}")
return None
def set_headers(self, url: str):
# use tcgplayerendpoints enum to set headers where url partially matches enum value
for endpoint in TCGPlayerEndpoints:
if endpoint.value in url:
return URLHeaders.ORDER_HEADERS
else:
raise ValueError(f"Endpoint not found in TCGPlayerEndpoints: {url}")
def old_set_headers(self, method: str) -> Dict:
base_headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
'accept-language': 'en-US,en;q=0.8',
'priority': 'u=0, i',
'referer': 'https://store.tcgplayer.com/admin/pricing',
'sec-ch-ua': '"Not A(Brand";v="8", "Chromium";v="132", "Brave";v="132"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'document',
'sec-fetch-mode': 'navigate',
'sec-fetch-site': 'same-origin',
'sec-fetch-user': '?1',
'sec-gpc': '1',
'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36'
}
if method == 'POST':
post_headers = {
'cache-control': 'max-age=0',
'content-type': 'application/x-www-form-urlencoded',
'origin': 'https://store.tcgplayer.com'
}
base_headers.update(post_headers)
return base_headers
"""
curl 'https://order-management-api.tcgplayer.com/orders/E576ED4C-38871F-B0277?api-version=2.0' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.8' \
-b 'tcgpartner=PK=TRADECARDS&M=1; valid=set=true; product-display-settings=sort=price+shipping&size=10; TCG_Data=M=1&SearchGameNameID=magic; tcg-uuid=ab16b5f8-dd66-446d-b217-d394328a5cf1; setting=CD=US&M=1; tracking-preferences={%22version%22:1%2C%22destinations%22:{%22Actions%20Amplitude%22:false%2C%22AdWords%22:false%2C%22Google%20AdWords%20New%22:false%2C%22Google%20Enhanced%20Conversions%22:false%2C%22Google%20Tag%20Manager%22:false%2C%22Impact%20Partnership%20Cloud%22:false%2C%22Optimizely%22:false}%2C%22custom%22:{%22advertising%22:false%2C%22functional%22:false%2C%22marketingAndAnalytics%22:false}}; OAuthLoginSessionId=63b1a89d-1ac2-43f7-8e79-55a9ca5e761d; TCGAuthTicket_Production=F453BFF2FA3FAA3D1ACA23F319314F8273713DECEB06C69BB7C208A77C81559B46E1AA22A7E70FABC8D7F681A423C86870FAE318B76048CDE7BF6D73D220631B899BEBA86C422E1EBBF2ACD1921E0846F708AFE203C844031364E13B047465E7B41CB6460E4F4AAB278B614445B93E722E976688; BuyerRevalidationKey=; ASP.NET_SessionId=oouwzrh3jkhdrmaioooqhr4k; TCG_VisitorKey=431efcca-2d5b-404d-a04f-3ae979696051; __RequestVerificationToken_L2FkbWlu0=Lw1sfWh823UeJ7zRux0b1ZTI4Vg4i_dFt97a55aQpf-qBURVuwWDCJyuCxSwgLNLe9nPlfDSc1AMV5nyqhY4Q4jurxs1; fileDownloadToken=1740145585435; spDisabledUIFeatures=' \
-H 'origin: https://sellerportal.tcgplayer.com' \
-H 'priority: u=1, i' \
-H 'referer: https://sellerportal.tcgplayer.com/' \
-H 'sec-ch-ua: "Not(A:Brand";v="99", "Brave";v="133", "Chromium";v="133"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'sec-gpc: 1' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'
"""

View File

@ -2,7 +2,7 @@ from fastapi.testclient import TestClient
from fastapi import BackgroundTasks from fastapi import BackgroundTasks
import pytest import pytest
import os import os
from main import app from app.main import app

View File

@ -4,9 +4,9 @@ import pytest
from unittest.mock import Mock, patch from unittest.mock import Mock, patch
import asyncio import asyncio
import os import os
from main import app from app.main import app
from services.file import FileService from app.services.file import FileService
from services.task import TaskService from app.services.task import TaskService
client = TestClient(app) client = TestClient(app)

6
dns.txt Normal file
View File

@ -0,0 +1,6 @@
@ IN MX 1 aspmx.l.google.com.
@ IN MX 5 alt1.aspmx.l.google.com.
@ IN MX 5 alt2.aspmx.l.google.com.
@ IN MX 10 alt3.aspmx.l.google.com.
@ IN MX 10 alt4.aspmx.l.google.com.
@ IN TXT "v=spf1 include:_spf.google.com ~all"

21
requests.md Normal file
View File

@ -0,0 +1,21 @@
curl -J http://192.168.1.41:8000/api/tcgplayer/inventory/update --remote-name
curl -J -X POST http://192.168.1.41:8000/api/tcgplayer/inventory/add \
-H "Content-Type: application/json" \
-d '{"open_box_ids": ["fcc28f9e-bc8b-4e89-a46e-7da746a96a5b","4a09930f-1830-4b8b-8006-a391e1adca66"]}' \
--remote-name
curl -X POST http://192.168.1.41:8000/api/boxes \
-F "type=collector" \
-F "set_code=TDM" \
-F "sku=1234" \
-F "num_cards_expected=123"
curl -X POST "http://192.168.1.41:8000/api/boxes/d95d26a8-1f82-47f2-89fa-3f88a4636823/open" \
-F "product_id=d95d26a8-1f82-47f2-89fa-3f88a4636823" \
-F "file_ids=0f29efd2-448c-4a05-ba49-1420fd3d524b" \
-F "date_opened=2025-04-04"
curl -X POST "http://192.168.1.41:8000/api/processOrders" \
-H "Content-Type: application/json" \
-d '{"order_ids": ["E576ED4C-A472E7-36237","E576ED4C-16EDF6-A340B","E576ED4C-A25B9C-A8855","E576ED4C-0E1A20-C6350","E576ED4C-9E8C21-3A249","E576ED4C-3825F0-5A5CC","E576ED4C-628925-6348B","E576ED4C-5F4314-6E3D2","E576ED4C-60E0B9-69D1D","E576ED4C-4BEC42-B2D0A","E576ED4C-5253F2-E2E16","E576ED4C-C08EA2-F51B4","E576ED4C-EE350E-BA82C","E576ED4C-CB067C-21150","E576ED4C-85DE3E-4E518","E576ED4C-27DB4A-A7729","E576ED4C-91A537-2AEBA","E576ED4C-3961D0-4F5A9","E576ED4C-EB7B7D-DBE0D","E576ED4C-1F9576-A9351","E576ED4C-7EBF1E-6FDB9","E576ED4C-F549E2-C558B","E576ED4C-215B45-4F177","E576ED4C-572FAA-004F7","E576ED4C-9D5F33-1A3C4","E576ED4C-87276B-63EC8","E576ED4C-2143E7-4DE1B","E576ED4C-41E56A-04D55","E576ED4C-789397-BF6AD","E576ED4C-2F3F46-154FE","E576ED4C-EFCBEE-3FE93","E576ED4C-3ADBAE-7CA1B","E576ED4C-D9F68F-A5E6F","E576ED4C-DEA6E2-8B590","E576ED4C-86D96B-DC5C4","E576ED4C-EDFABA-67C3C","E576ED4C-C57373-3F638","E576ED4C-B2C2B4-FF53B","E576ED4C-3788E5-B3653","E576ED4C-8A573A-BB51B","E576ED4C-497380-63F5C","E576ED4C-A6C3F2-C7BF2","E576ED4C-FAC80B-148F3","E576ED4C-ECF1F3-AF3A4"]}'

View File

@ -1,18 +1,32 @@
alembic==1.14.1
annotated-types==0.7.0 annotated-types==0.7.0
anyio==4.8.0 anyio==4.8.0
APScheduler==3.11.0
browser-cookie3==0.20.1 browser-cookie3==0.20.1
certifi==2025.1.31 certifi==2025.1.31
charset-normalizer==3.4.1 charset-normalizer==3.4.1
click==8.1.8 click==8.1.8
coverage==7.6.10
fastapi==0.115.8 fastapi==0.115.8
h11==0.14.0 h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
idna==3.10 idna==3.10
iniconfig==2.0.0
lz4==4.4.3 lz4==4.4.3
Mako==1.3.9
MarkupSafe==3.0.2
numpy==2.2.2 numpy==2.2.2
packaging==24.2
pandas==2.2.3 pandas==2.2.3
pluggy==1.5.0
psycopg2-binary==2.9.10
pycryptodomex==3.21.0 pycryptodomex==3.21.0
pydantic==2.10.6 pydantic==2.10.6
pydantic_core==2.27.2 pydantic_core==2.27.2
pytest==8.3.4
pytest-asyncio==0.25.3
pytest-cov==6.0.0
python-dateutil==2.9.0.post0 python-dateutil==2.9.0.post0
python-multipart==0.0.20 python-multipart==0.0.20
pytz==2025.1 pytz==2025.1
@ -23,5 +37,6 @@ SQLAlchemy==2.0.37
starlette==0.45.3 starlette==0.45.3
typing_extensions==4.12.2 typing_extensions==4.12.2
tzdata==2025.1 tzdata==2025.1
tzlocal==5.2
urllib3==2.3.0 urllib3==2.3.0
uvicorn==0.34.0 uvicorn==0.34.0

View File

@ -1,19 +0,0 @@
from pydantic import BaseModel, Field, ConfigDict
from typing import Optional
from datetime import datetime
# FILE
class OrderSchema(BaseModel):
id: str = Field(..., title="id")
filename: str = Field(..., title="filename")
type: str = Field(..., title="type")
filesize_kb: float = Field(..., title="filesize_kb")
source: str = Field(..., title="source")
status: str = Field(..., title="status")
service: Optional[str] = Field(None, title="service")
date_created: datetime = Field(..., title="date_created")
date_modified: datetime = Field(..., title="date_modified")
# This enables ORM mode
model_config = ConfigDict(from_attributes=True)

38
send_cookie.py Normal file
View File

@ -0,0 +1,38 @@
import browser_cookie3
import requests
import json
def send_tcg_cookies(api_url: str, browser_type='brave'):
"""Get TCGPlayer cookies and send them to the API"""
try:
# Get cookies from browser
cookie_getter = getattr(browser_cookie3, browser_type)
cookie_jar = cookie_getter(domain_name='tcgplayer.com')
# Filter essential cookies
cookies = {}
for cookie in cookie_jar:
if any(key in cookie.name.lower() for key in ['.aspnet', 'tcg', 'session']):
cookies[cookie.name] = cookie.value
# Send to API
headers = {
'Content-Type': 'application/json'
}
response = requests.post(
f"{api_url}",
headers=headers,
json={'cookies': cookies}
)
response.raise_for_status()
print("Cookies updated successfully!")
except Exception as e:
print(f"Error updating cookies: {e}")
if __name__ == "__main__":
API_URL = "http://192.168.1.41:8000/api/cookies" # Update with your API URL
send_tcg_cookies(API_URL)