mirror of
https://gitlab.com/jrmiller82/pathfinder-2-sqlite.git
synced 2026-10-12 22:29:44 -04:00
MAJOR DIRECTORY OVERHAUL
This commit is contained in:
jrmiller82 committed
1 parent
71ed03b165
commit
34ef5a6025
138 files changed
+48
-6
No files matched your search
@@ -0,0 +1,9 @@
|
||||
# The purpose of this directory is to perform scraping of [Archives of Nethys](https://2e.aonprd.com)
|
||||
|
||||
## Requirements
|
||||
1. Python 3.6.8
|
||||
2. pandas==0.24.2
|
||||
3. splinter==0.11.0
|
||||
4. beautifulsoup4==4.8.0
|
||||
5. selenium==3.141.0
|
||||
6. Download the [chrome WebDriver](https://splinter.readthedocs.io/en/latest/drivers/chrome.html) and place it in this directory. It is required by selenium and splinter as using Requests was prone to failure.
|
||||
@@ -0,0 +1,51 @@
|
||||
Name,description
|
||||
Acolyte,"You spent your early days in a religious monastery or cloister. You may have traveled out into the world to spread the message of your religion or because you cast away the teachings of your faith, but deep down you’ll always carry within you the lessons you learned. \n\nChoose two ability boosts. One must be to Intelligence or Wisdom, and one is a free ability boost. \n\nYou're trained in the Religion skill and the Scribing Lore skill. You gain the Student of the Canon skill feat."
|
||||
Acrobat,"In a circus or on the streets, you earned your pay by performing as an acrobat. You might have turned to adventuring when the money dried up, or simply decided to put your skills to better use. \n\nChoose two ability boosts. One must be to Strength or Dexterity, and one is a free ability boost. \n\nYou're trained in the Acrobatics skill and the Circus Lore skill. You gain the Steady Balance skill feat."
|
||||
Animal Whisperer,"You have always felt a connection to animals, and it was only a small leap to learn to train them. As you travel, you continuously encounter different creatures, befriending them along the way. \n\nChoose two ability boosts. One must be to Wisdom or Charisma, and one is a free ability boost. \n\nYou're trained in the Nature skill and a Lore skill related to one terrain inhabited by animals you like (such as Plains Lore or Swamp Lore). You gain the Train Animal skill feat."
|
||||
Artisan,"As an apprentice, you practiced a particular form of building or crafting, developing specialized skill. You might have been a blacksmith’s apprentice toiling over the forge for countless hours, a young tailor sewing garments of all kinds, or a shipwright shaping the hulls of ships. \n\nChoose two ability boosts. One must be to Strength or Intelligence, and one is a free ability boost. \n\nYou're trained in the Crafting skill and the Guild Lore skill. You gain the Specialty Crafting skill feat."
|
||||
Artist,"Your art is your greatest passion, whatever form it takes. Adventuring might help you find inspiration, or simply be a way to survive until you become a world-famous artist. \n\nChoose two ability boosts. One must be to Dexterity or Charisma, and one is a free ability boost. \n\nYou're trained in the Crafting skill and the Art Lore skill. You gain the Specialty Crafting skill feat."
|
||||
Barkeep,"You have five specialties: hefting barrels, drinking, polishing steins, drinking, and drinking. You worked in a bar, where you learned how to hold your liquor and rowdily socialize. \n\nChoose two ability boosts. One must be to Constitution or Charisma, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and the Alcohol Lore skill. You gain the Hobnobber skill feat."
|
||||
Barrister,"Piles of legal manuals, stern teachers, and experience in the courtroom have instructed you in legal matters. You’re capable of mounting a prosecution or defense in court, and you tend to keep abreast of local laws, as you never can tell when you might need to know them on short notice. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and the Legal Lore skill. You gain the Group Impression skill feat."
|
||||
Bounty Hunter,"Bringing in lawbreakers lined your pockets. Maybe you had an altruistic motive and sought to bring in criminals to make the streets safer, or maybe the coin was motivation enough. Your techniques for hunting down criminals transfer easily to the life of an adventurer. \n\nChoose two ability boosts. One must be to Strength or Wisdom, and one is a free ability boost. \n\nYou're trained in the Survival skill and the Legal Lore skill. You gain the Experienced Tracker skill feat."
|
||||
Charlatan,"You traveled from place to place, peddling false fortunes and snake oil in one town, pretending to be royalty in exile to seduce a wealthy heir in the next. Becoming an adventurer might be your next big scam or an attempt to put your talents to use for a greater cause. Perhaps it’s a bit of both, as you realize that after pretending to be a hero, you’ve become the mask. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Deception skill and the Underworld Lore skill. You gain the Charming Liar skill feat."
|
||||
Criminal,"As an unscrupulous independent or as a member of an underworld organization, you lived a life of crime. You might have become an adventurer to seek redemption, to escape the law, or simply to get access to bigger and better loot. \n\nChoose two ability boosts. One must be to Dexterity or Intelligence, and one is a free ability boost. \n\nYou're trained in the Stealth skill and the Underworld Lore skill. You gain the Experienced Smuggler skill feat."
|
||||
Detective,"You solved crimes as a police inspector or took jobs for wealthy clients as a private investigator. You might have become an adventurer as part of your next big mystery, but likely it was due to the consequences or aftermath of a prior case. \n\nChoose two ability boosts. One must be to Intelligence or Wisdom, and one is a free ability boost. \n\nYou're trained in the Society skill and the Underworld Lore skill. You gain the Streetwise skill feat."
|
||||
Emissary,"As a diplomat or messenger, you traveled to lands far and wide. Communicating with new people and forming alliances were your stock and trade. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Society skill and a Lore skill related to one city you've visited often. You gain the Multilingual skill feat."
|
||||
Entertainer,"Through an education in the arts or sheer dogged practice, you learned to entertain crowds. You might have been an actor, a dancer, a musician, a street magician, or any other sort of performer. \n\nChoose two ability boosts. One must be to Dexterity or Charisma, and one is a free ability boost. \n\nYou're trained in the Performance skill and the Theater Lore skill. You gain the Fascinating Performance skill feat."
|
||||
Farmhand,"With a strong back and an understanding of seasonal cycles, you tilled the land and tended crops. Your farm could have been razed by invaders, you could have lost the family tying you to the land, or you might have simply tired of the drudgery, but at some point you became an adventurer. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Athletics skill and the Farming Lore skill. You gain the Assurance skill feat with Athletics."
|
||||
Field Medic,"In the chaotic rush of battle, you learned to adapt to rapidly changing conditions as you administered to battle casualties. You patched up soldiers, guards, or other combatants, and learned a fair amount about the logistics of war. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Medicine skill and the Warfare Lore skill. You gain the Battle Medicine skill feat."
|
||||
Fortune Teller,"The strands of fate are clear to you, as you have learned many traditional forms by which laypeople can divine the future. You might have used these skills to guide your community, or simply to make money. But even the slightest peek into these practices connects you to the occult mysteries of the universe. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Occultism skill and the Fortune-Telling Lore skill. You gain the Oddity Identification skill feat."
|
||||
Gambler,"The thrill of the win drew you into games of chance. This might have been a lucrative sideline that paled in comparison to the real risks of adventuring, or you might have fallen on hard times due to your gambling and pursued adventuring as a way out of a spiral. \n\nChoose two ability boosts. One must be to Dexterity or Charisma, and one is a free ability boost. \n\nYou're trained in the Deception skill and the Games Lore skill. You gain the Lie to Me skill feat."
|
||||
Gladiator,"The bloody games of the arena taught you the art of combat. Before you attained true fame, you departed—or escaped—the arena to explore the world. Your skill at drawing both blood and a crowd’s attention pay off in a new adventuring life. \n\nChoose two ability boosts. One must be to Strength or Charisma, and one is a free ability boost. \n\nYou're trained in the Performance skill and the Gladiatorial Lore skill. You gain the Impressive Performance skill feat."
|
||||
Guard,"You served in the guard, out of either patriotism or the need for coin. Either way, you know how to get a difficult suspect to talk. However you left the guard, you might think of adventuring as a way to use your skills on a wider stage. \n\nChoose two ability boosts. One must be to Strength or Charisma, and one is a free ability boost. \n\nYou're trained in the Intimidation skill and the Legal Lore skill or Warfare Lore skill. You gain the Quick Coercion skill feat."
|
||||
Herbalist,"As a formally trained apothecary or a rural practitioner of folk medicine, you learned the healing properties of various herbs. You’re adept at collecting the right natural cures in all sorts of environments and preparing them properly. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Nature skill and the Herbalism Lore skill. You gain the Natural Medicine skill feat."
|
||||
Hermit,"In an isolated place—like a cave, remote oasis, or secluded mansion—you lived a life of solitude. Adventuring might represent your first foray out among other people in some time. This might be a welcome reprieve from solitude or an unwanted change, but in either case, you’re likely still rough around the edges. \n\nChoose two ability boosts. One must be to Constitution or Intelligence, and one is a free ability boost. \n\nYou're trained in your choice of either the Nature or Occultism skill, as well as a Lore skill related to the terrain you lived in as a hermit (such as Cave Lore or Desert Lore). You gain the Dubious Knowledge skill feat."
|
||||
Hunter,"You stalked and took down animals and other creatures of the wild. Skinning animals, harvesting their flesh, and cooking them were also part of your training, all of which can give you useful resources while you adventure. \n\nChoose two ability boosts. One must be to Dexterity or Wisdom, and one is a free ability boost. \n\nYou're trained in the Survival skill and the Tanning Lore skill. You gain the Survey Wildlife skill feat."
|
||||
Laborer,"You’ve spent years performing arduous physical labor. It was a difficult life, but you somehow survived. You may have embraced adventuring as an easier method to make your way in the world, or you might adventure under someone else’s command. \n\nChoose two ability boosts. One must be to Strength or Constitution, and one is a free ability boost. \n\nYou're trained in the Athletics skill and the Labor Lore skill. You gain the Hefty Hauler skill feat in Athletics."
|
||||
Martial Disciple,"You dedicated yourself to intense training and rigorous study to become a great warrior. The school you attended might have been a traditionalist monastery, an elite military academy, or the local branch of a prestigious mercenary organization. \n\nChoose two ability boosts. One must be to Strength or Dexterity, and one is a free ability boost. \n\nYou're trained in your choice of either the Acrobatics or Athletics skill, as well as the Warfare Lore skill. You gain a skill feat: Cat Fall if you chose Acrobatics or Quick Jump if you chose Athletics."
|
||||
Merchant,"In a dusty shop, market stall, or merchant caravan, you bartered wares for coin and trade goods. The skills you picked up still apply in the adventuring life, in which a good deal on a suit of armor could prevent your death. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and the Mercantile Lore skill. You gain the Bargain Hunter skill feat."
|
||||
Miner,"You earned a living wrenching precious minerals from the lightless depths of the earth. Adventuring might have seemed lucrative or glamorous compared to this backbreaking labor— and if you have to head back underground, this time you plan to do so armed with a real weapon instead of a miner’s pick. \n\nChoose two ability boosts. One must be to Strength or Wisdom, and one is a free ability boost. \n\nYou're trained in the Survival skill and the Mining Lore skill. You gain the Terrain Expertise skill feat with underground terrain."
|
||||
Noble,"To the common folk, the life of a noble seems one of idyllic luxury, but growing up as a noble or member of the aspiring gentry, you know the reality: a noble’s lot is obligation and intrigue. Whether you seek to escape your duties by adventuring or to better your station, you have traded silks and pageantry for an adventurer’s life. \n\nChoose two ability boosts. One must be to Intelligence or Charisma, and one is a free ability boost. \n\nYou're trained in the Society skill and the Genealogy Lore skill or Heraldy Lore skill. You gain the Courtly Graces skill feat."
|
||||
Nomad,"Traveling far and wide, you picked up basic tactics for surviving on the road and in unknown lands, getting by with few supplies and even fewer comforts. As an adventurer, you travel still, often into even more dangerous places. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Survival skill and a Lore skill related to one terrain you traveled in (such as Desert Lore or Swamp Lore). You gain the Assurance skill feat with Survival."
|
||||
Prisoner,"You might have been imprisoned for crimes (whether you were guilty or not), or enslaved for some part of your upbringing. In your adventuring life, you take full advantage of your newfound freedom. \n\nChoose two ability boosts. One must be to Strength or Constitution, and one is a free ability boost. \n\nYou're trained in the Stealth skill and the Underworld Lore skill. You gain the Experienced Smuggler skill feat."
|
||||
Sailor,"You heard the call of the sea from a young age. Perhaps you signed onto a merchant’s vessel, joined the navy, or even fell in with a crew of pirates and scalawags. \n\nChoose two ability boosts. One must be to Strength or Dexterity, and one is a free ability boost. \n\nYou're trained in the Athletics skill and the Sailing Lore skill. You gain the Underwater Marauder skill feat."
|
||||
name: 31,name: 31
|
||||
Scout,"You called the wilderness home as you found trails and guided travelers. Your wanderlust could have called you to the adventuring life, or perhaps you served as a scout for soldiers and found you liked battle. \n\nChoose two ability boosts. One must be to Dexterity or Wisdom, and one is a free ability boost. \n\nYou're trained in the Survival skill and a Lore skill related to one terrain you scouted in (such as Forest Lore or Cavern Lore). You gain the Forager skill feat."
|
||||
Street Urchin,"You eked out a living by picking pockets on the streets of a major city, never knowing where you’d find your next meal. While some folk adventure for the glory, you do so to survive. \n\nChoose two ability boosts. One must be to Dexterity or Constitution, and one is a free ability boost. \n\nYou're trained in the Thievery skill and a Lore skill for the city you lived in as a street urchin (such as Absalom Lore or Magnimar Lore). You gain the Pickpocket skill feat."
|
||||
Tinker,"Creating all sorts of minor inventions scratches your itch for problem-solving. Your engineering skills take a particularly creative bent, and no one know what you’ll come up with next. It might be a genius device with tremendous potential... or it might explode. \n\nChoose two ability boosts. One must be to Dexterity or Intelligence, and one is a free ability boost. \n\nYou're trained in the Crafting skill and the Engineering Lore skill. You gain the Specialty Crafting skill feat."
|
||||
Warrior,"In your younger days, you waded into battle as a mercenary, a warrior defending a nomadic people, or a member of a militia or army. You might have wanted to break out from the regimented structure of these forces, or you could have always been as independent a warrior as you are now. \n\nChoose two ability boosts. One must be to Strength or Constitution, and one is a free ability boost. \n\nYou're trained in the Intimidation skill and the Warfare Lore skill. You gain the Intimidating Glare skill feat."
|
||||
Scholar,"You have a knack for learning, and sequestered yourself from the outside world to learn all you could. You read about so many wondrous places and things in your books, and always dreamed about one day seeing the real things. Eventually, that curiosity led you to leave your studies and become an adventurer. \n\nChoose two ability boosts. One must be to Intelligence or Wisdom, and one is a free ability boost. \n\nYou’re trained in your choice of the Arcana, Nature, Occultism, or Religion skill, and the Academia Lore skill. You gain the Assurance skill feat."
|
||||
Lesser Scion,"You are the youngest child in a noble house (in Cheliax, Isger, or Andoran) and stand to inherit nothing from your family. Although you have a minor title, it affords you no lands or wealth, but it has garnered you a small amount of respect and deference in your travels. When play begins, you are riding in a caravan bound for Almas, where a cousin has promised to allow you to stay with them for a month. \n\nChoose two ability boosts. One must be to Dexterity or Intelligence, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and the Heraldry Lore skill. You gain the Hobnobber skill feat."
|
||||
Lost and Alone,"You were training to become a knight in Lastwall when the Whispering Tyrant escaped his imprisonment and destroyed the nation. It was only by dumb luck that you are alive at all, but the memories of that fateful day haunt your dreams. When play begins, you have boarded a caravan heading toward a new town, having worn out your welcome at the taverns and inns of Elidir. \n\nChoose two ability boosts. One must be to Strength or Charisma, and one is a free ability boost. \n\nYou're trained in the Intimidation skill and the Warfare Lore skill. You gain the Intimidating Glare skill feat."
|
||||
Missionary,"You received training from clergy in a faraway temple, who sent you out into the world to spread the faith. Although you are relatively new at this, you are always on the lookout for new places in need of your deity’s teachings and guidance. When play begins, you are riding along in a caravan making your way through sparsely populated regions of Isger, spreading the news of your faith to those who will listen. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and the Scribing Lore skill. You gain the Group Impression skill feat."
|
||||
Refugee,"The fighting around the Lake Encarthan region has forced many of the people residing there to flee from the violence. Although your home is gone, you have managed to survive and are making your way south in search of a better life. When play begins, you are riding along in a caravan taking you to what might be either a new home in Andoran or simply another waypoint in your journey. \n\nChoose two ability boosts. One must be to Constitution or Intelligence, and one is a free ability boost. \n\nYou're trained in the Survival skill and the Hunting Lore skill. You gain the Forager skill feat."
|
||||
Teamster,"You left your home a few months back for a life on the road, working for one caravan company and then another, always on the lookout for a new job and a better life. Last week, you were hired by the Bort Bargith’s company in Elidir. You don’t know anyone from the company just yet, but most of its members seem to be honest merchants and traders. \n\nChoose two ability boosts. One must be to Strength or Wisdom, and one is a free ability boost. \n\nYou're trained in the Nature skill and the Mercantile Lore skill. You gain the Train Animal skill feat."
|
||||
Dragon Scholar,"Dragons have fascinated you for as long as you can remember. The potent power they possess, the long lives they lead, the ancient traditions they inspire—whatever the source of your interest, you’ve spent much of your life to this point immersed in all things draconic. These studies have bolstered your self-confidence and given you a wide range of methods and tactics you can use to impose your will on others. \n\n You’ve likely chosen to attend the Call for Heroes as a way to seek funds as an adventurer to afford more texts about dragons or perhaps out of a desire to encounter one in real life. \n\nChoose two ability boosts. One must be to Strength or Charisma, and one is a free ability boost. \n\nYou're trained in the Intimidation skill and the Dragon Lore skill. You gain the Intimidating Glare skill feat."
|
||||
Emancipated,"Until recently, the nation of Ravounel was part of the larger nation of Cheliax, where the church of Asmodeus is the law and slaves are traded freely in the open market. You had the poor fortune to be born into slavery, but the good luck to have grown up in the city of Kintargo. When Ravounel seceded from Cheliax, the leaders of this new nation freed all slaves, and you’ve wasted no time in exploring and establishing your new life. How and why you’ve come to Breachill is left to you to decide—but the fact that you feel empowered to determine your own destiny continues to drive you! \n\n The chance to become an adventurer has excited you for some time, as you hope to build a new life for yourself as a hero rewarded with fame and fortune. Joining the Call for Heroes is a great opportunity to find a group to adventure with. \n\nChoose two ability boosts. One must be to Dexterity or Charisma, and one is a free ability boost. \n\nYou're trained in the Stealth skill and the Kintargo Lore skill. You gain the Terrain Stalker skill feat."
|
||||
Haunting Vision,"You’ve been haunted by frightening dreams of fires your whole life. This could be the result of a past, neardeath experience with fire or it might have no obvious source that you know of. Recently, you stumbled upon an image, story, or other omen featuring the dragon god of destruction, Dahak, and you were struck with an unnerving sense of deja vu—you’ve come to think that the draconic deity might have something to do with your dreams, and as such have been studying all you can about the god. Your visions have bolstered your faith as well; even if you don’t worship a specific deity, you have a deep passion for matters of faith. \n\n Your latest dream, for the first time, had enough details to identify its setting—in this last dream, you saw the town of Breachill burning. You’ve decided to join the Call for Heroes hoping to save the town you fear might burn to the ground soon. \n\nChoose two ability boosts. One must be to Constitution or Wisdom, and one is a free ability boost. \n\nYou're trained in the Religion skill and the Dahak Lore skill. You gain the Student of the Canon skill feat."
|
||||
Hellknight Historian,"The various Hellknight Orders intrigue you, whether you seek to become one of their number yourself, wish to oppose their goals at every turn, or are merely inspired by or curious to learn more about them. You’ve heard that when the Order of the Nail abandoned Citadel Altaerein, they left behind the deed to the castle so anyone brave enough to explore the ruins and the defenses no doubt left behind would be rewarded with ownership of the abandoned fortress. \n\n You decided to join the Call for Heroes to hoping to gain an opportunity to explore Citadel Altaerein. \n\nChoose two ability boosts. One must be to Strength or Intelligence, and one is a free ability boost. \n\nYou're trained in the Society skill and the Architecture Lore skill. You gain the Courtly Graces skill feat."
|
||||
Local Scion,"You’re from Breachill and have lived there most, if not all, of your life. You might be the son or daughter of a wellknown local adventuring family, or a family with a storied tradition of military or other martial service. You likely also have some sort of skill with your hands, as the people of Breachill are very self-sufficient. \n\n The Call for Heroes is one of your hometown’s most iconic traditions, and you want to attend so you can prove your own merits to the council—beyond simply those of your family’s name. \n\nChoose two ability boosts. One must be to Constitution or Charisma, and one is a free ability boost. \n\nYou're trained in the Crafting skill and the Breachill Lore skill. You gain the Specialty Crafting skill feat."
|
||||
Out-of-Towner,"You’ve never been to Breachill, nor do you have roots in the town, but as chance has it you find yourself visiting. You might be here traveling with a friend, visiting an old acquaintance, or merely here to see a new part of the world. Before coming to town, though, you spent many years living with an ancestry other than your own, and your diverse childhood has left you with an open mind and a curious nature. \n\n You’re excited to meet new people and cultures, and answering the Call for Heroes seems to be an interesting way to do so. \n\nChoose two ability boosts. One must be to Constitution or Intelligence, and one is a free ability boost. \n\nYou're trained in the Diplomacy skill and a Lore skill thematically associated with the members of the ancestry you grew up with: Dwarf, Elf, Gnome, Goblin, or Halfling. You gain the Hobnobber skill feat."
|
||||
Reputation Seeker,"You’re likely from Breachill, but unlike the Local Scion, your family has no notable legacy in the area. You might not even have much of a family at all to call your own, and could be an orphan. You’ve seen so many people make names for themselves that you set out on your own and spent some time abroad in the jungles or deserts of Garund or in the upper reaches of the Darklands—regions that proved too dangerous to remain in for long on your own. You returned home with more caution and knowledge of the world beyond Breachill’s borders, but still determined to win fame. \n\n You’ve decided that joining an adventuring group would be the best way to secure aid in your quest to build your own reputation, and are attending the Call for Heroes to find such allies. \n\nChoose two ability boosts. One must be to Dexterity or Intelligence, and one is a free ability boost. \n\nYou're trained in the Survival skill and the Darklands, Desert, or Jungle Lore skill. You gain the Terrain Expertise skill feat (underground if you have Darklands Lore, desert if you have Desert Lore, or forest if you have Jungle Lore)."
|
||||
Returning Descendant,"You aren’t from Breachill yourself—you might not even be from Isger or Avistan at all. You’ve lived a hard life, in any event. You have often had to make tough decisions, and have developed a knack for understanding how things work (and the best way to take them apart) that has helped you support yourself. Recently you discovered that an ancestor was from the town of Breachill and had a modest career as an adventurer. You can’t help but wonder if you would have had an easier life if you’d grown up there instead, and have decided to seek out this small town to learn more about it. \n\nYou’ve decided to respond to the Call for Heroes to follow in your ancestor’s footsteps, either to honor their memory or accomplish the great deeds they did not. \n\nChoose two ability boosts. One must be to Dexterity or Wisdom, and one is a free ability boost. \n\nYou're trained in the Thievery skill and the Engineering Lore skill. You gain the Pickpocket skill feat."
|
||||
Truth Seeker,"Whether you are a local or from out of town, you’ve heard rumors that Breachill’s past contains a hidden secret. Perhaps you’ve heard strange rumors that the town’s founder, Lamond Breachton, was not the hero everyone touts, or maybe your grandmother heard stories from her own grandmother that contradict the town’s accepted narrative of its establishment. In the pursuit of the truth, you’ve learned to navigate the tangles of politics, and to never take anyone’s word at face value. \n\n You plan to join the Call for Heroes so that you can make yourself known to the council, or perhaps even ingratiating yourself to them, so you can seek the truth and eventually uncover Breachill’s secrets! \n\nChoose two ability boosts. One must be to Strength or Wisdom, and one is a free ability boost. \n\nYou're trained in the Deception skill and the Politics Lore skill. You gain the Lie to Me skill feat."
|
||||
|
@@ -0,0 +1,152 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Scrape data from aon2e and generate csvs to import in to sqlite"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Dependencies\n",
|
||||
"import pandas as pd\n",
|
||||
"from bs4 import BeautifulSoup as bs, Tag, NavigableString\n",
|
||||
"import os\n",
|
||||
"from splinter import Browser\n",
|
||||
"\n",
|
||||
"# Setting up Selenium\n",
|
||||
"chrome_driver = os.path.join('..', 'chromedriver.exe')\n",
|
||||
"executable_path = {'executable_path': chrome_driver}\n",
|
||||
"browser = Browser('chrome', **executable_path, headless=False)\n",
|
||||
"\n",
|
||||
"# Pandas config\n",
|
||||
"pd.set_option('display.max_columns', None)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def scrape_description(url, id_number):\n",
|
||||
"\n",
|
||||
" # Empty lists to store the scraped values\n",
|
||||
" name_list = []\n",
|
||||
" description_list = []\n",
|
||||
"\n",
|
||||
" print(f'Beginning Data Retrieval')\n",
|
||||
" print(f'------------------------')\n",
|
||||
"\n",
|
||||
" # Loop from 1 to the value in weapon_number\n",
|
||||
" for page in range(1, id_number+1):\n",
|
||||
"\n",
|
||||
" browser.visit(url + str(page))\n",
|
||||
" html = browser.html\n",
|
||||
" soup = bs(html, 'html.parser')\n",
|
||||
"\n",
|
||||
" # Select only the content section\n",
|
||||
" content = soup.find(id='ctl00_MainContent_DetailedOutput')\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # Store the name\n",
|
||||
" name = content.find('h1', class_='title').a.text.strip()\n",
|
||||
" name\n",
|
||||
"\n",
|
||||
" except:\n",
|
||||
" name = f'name: {page}'\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # Start the loop after the link to the book\n",
|
||||
" start = content.find('a', class_='external-link').next_sibling\n",
|
||||
" description = ''\n",
|
||||
" for e in start.next_siblings:\n",
|
||||
" if isinstance(e, Tag):\n",
|
||||
" if e.name == 'br':\n",
|
||||
" if e.next_sibling.name == 'br':\n",
|
||||
" # If the next 2 elements are br skip this\n",
|
||||
" # loop it will be handled in the elif\n",
|
||||
" continue\n",
|
||||
" elif e.previous_sibling.name == 'br':\n",
|
||||
" # If this element and the previous are br\n",
|
||||
" # and the next is not append /n\n",
|
||||
" description = description + ' /n/n'\n",
|
||||
" else:\n",
|
||||
" # If there is just one br append /n\n",
|
||||
" description = description + ' /n'\n",
|
||||
" else:\n",
|
||||
" # Append the text inside the element\n",
|
||||
" description = description + e.text.strip()\n",
|
||||
" elif isinstance(e, NavigableString):\n",
|
||||
" # Since it is just a text append it\n",
|
||||
" description = description + e\n",
|
||||
"\n",
|
||||
" except:\n",
|
||||
" description = f'name: {page}'\n",
|
||||
"\n",
|
||||
" print(f'{page} of {id_number} | {name}')\n",
|
||||
"\n",
|
||||
" # Append values to our empty lists\n",
|
||||
" name_list.append(name) \n",
|
||||
" description_list.append(description)\n",
|
||||
"\n",
|
||||
" print(f'------------------------')\n",
|
||||
" print(f'Data Retrieval Complete')\n",
|
||||
" \n",
|
||||
" # Create df with the scraped data\n",
|
||||
" data = {'Name': name_list, 'description': description_list}\n",
|
||||
" \n",
|
||||
" # Returns a data frame\n",
|
||||
" return pd.DataFrame(data)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# scrape the descriptions\n",
|
||||
"url_background = 'https://2e.aonprd.com/Backgrounds.aspx?ID='\n",
|
||||
"number_background = 50 # number to scrape\n",
|
||||
"\n",
|
||||
"description_background = scrape_description(url_background, number_background)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"description_background.to_csv('background.csv', encoding='UTF-8', index=False)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# # Scrape data from aon2e and generate csvs to import in to sqlite
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Dependencies
|
||||
import pandas as pd
|
||||
from bs4 import BeautifulSoup as bs, Tag, NavigableString
|
||||
import os
|
||||
from splinter import Browser
|
||||
|
||||
# Setting up Selenium
|
||||
chrome_driver = os.path.join('..', 'chromedriver.exe')
|
||||
executable_path = {'executable_path': chrome_driver}
|
||||
browser = Browser('chrome', **executable_path, headless=False)
|
||||
|
||||
# Pandas config
|
||||
pd.set_option('display.max_columns', None)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
def scrape_description(url, id_number):
|
||||
|
||||
# Empty lists to store the scraped values
|
||||
name_list = []
|
||||
description_list = []
|
||||
|
||||
print(f'Beginning Data Retrieval')
|
||||
print(f'------------------------')
|
||||
|
||||
# Loop from 1 to the value in weapon_number
|
||||
for page in range(1, id_number+1):
|
||||
|
||||
browser.visit(url + str(page))
|
||||
html = browser.html
|
||||
soup = bs(html, 'html.parser')
|
||||
|
||||
# Select only the content section
|
||||
content = soup.find(id='ctl00_MainContent_DetailedOutput')
|
||||
|
||||
try:
|
||||
# Store the name
|
||||
name = content.find('h1', class_='title').a.text.strip()
|
||||
name
|
||||
|
||||
except:
|
||||
name = f'name: {page}'
|
||||
|
||||
try:
|
||||
# Start the loop after the link to the book
|
||||
start = content.find('a', class_='external-link').next_sibling
|
||||
description = ''
|
||||
for e in start.next_siblings:
|
||||
if isinstance(e, Tag):
|
||||
if e.name == 'br':
|
||||
if e.next_sibling.name == 'br':
|
||||
# If the next 2 elements are br skip this
|
||||
# loop it will be handled in the elif
|
||||
continue
|
||||
elif e.previous_sibling.name == 'br':
|
||||
# If this element and the previous are br
|
||||
# and the next is not append /n
|
||||
description = description + ' /n/n'
|
||||
else:
|
||||
# If there is just one br append /n
|
||||
description = description + ' /n'
|
||||
else:
|
||||
# Append the text inside the element
|
||||
description = description + e.text.strip()
|
||||
elif isinstance(e, NavigableString):
|
||||
# Since it is just a text append it
|
||||
description = description + e
|
||||
|
||||
except:
|
||||
description = f'name: {page}'
|
||||
|
||||
print(f'{page} of {id_number} | {name}')
|
||||
|
||||
# Append values to our empty lists
|
||||
name_list.append(name)
|
||||
description_list.append(description)
|
||||
|
||||
print(f'------------------------')
|
||||
print(f'Data Retrieval Complete')
|
||||
|
||||
# Create df with the scraped data
|
||||
data = {'Name': name_list, 'description': description_list}
|
||||
|
||||
# Returns a data frame
|
||||
return pd.DataFrame(data)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# scrape the descriptions
|
||||
url_background = 'https://2e.aonprd.com/Backgrounds.aspx?ID='
|
||||
number_background = 50 # number to scrape
|
||||
|
||||
description_background = scrape_description(url_background, number_background)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
description_background.to_csv('background.csv', encoding='UTF-8', index=False)
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
# This directory scrapes the weapons from the [Archives of Nethys](https://2e.aonprd.com/Weapons.aspx)
|
||||
|
||||
## Steps to scrape the weapons
|
||||
1. Install the requirements from [the previous readme](../README.md)
|
||||
2. Generate .csv files from copy pasting the tables from [here](https://2e.aonprd.com/Weapons.aspx) and save them in this directory
|
||||
3. Set the number_of_weapons variable to the number of weapons in the database it is currently 83
|
||||
4. Run the [python file](scrape.py) or [Jupyter Notebook](scrape.ipynb)
|
||||
@@ -0,0 +1,105 @@
|
||||
equipment_id,name,level,price,bulk,traits,hands,source_id,source_pg,description
|
||||
1,Holly and Mistletoe,0,0,0,,1,1,290,"Plants of supernatural significance provide a primal focus for primal spellcasters, such as druids, when using certain abilities and casting some spells. A bundle of holly and mistletoe must be held in one hand to use it. Other primal foci exist for druids focused on other aspects of nature."
|
||||
2,Bedroll,0,0.01,0.1,,,1,288,
|
||||
3,Candle,0,0.01,0,,1,1,289,A lit candle sheds dim light in a 10-foot radius.
|
||||
4,Chalk,0,0.01,0,,1,1,288,
|
||||
5,Mug,0,0.01,0,,1,1,288,
|
||||
6,Oil,0,0.01,0,,2,1,291,"You can use oil to fuel lanterns, but you can also set a pint of oil aflame and throw it. You must first spend an Interact action preparing the oil, then throw it with another action as a ranged attack. If you hit, it splatters on the creature or in a single 5-foot square you target. You must succeed at a DC 10 flat check for the oil to ignite successfully when it hits. If the oil ignites, the target takes 1d6 fire damage."
|
||||
7,Piton,0,0.01,0,,1,1,291,"These small spikes can be used as anchors to make climbing easier. To affix a piton, you must hold it in one hand and use a hammer to drive it in with your other hand. You can attach a rope to the hammered piton so that you don't fall all the way to the ground on a critical failure while Climbing."
|
||||
8,Sack,0,0.01,0.1,,1,1,291,"A sack can hold up to 8 Bulk worth of items. A sack containing 2 Bulk or less can be worn on the body, usually tucked into a belt. You can carry a sack with one hand, but must use two hands to transfer items in and out."
|
||||
9,Sheath,0,0.01,0,,,1,291,A sheath or scabbard lets you easily carry a weapon on your person.
|
||||
10,Ten-Foot Pole,0,0.01,1,,2,1,291,"When wielding this long pole, you can use Seek to search a square up to 10 feet away. The pole is not sturdy enough to use as a weapon."
|
||||
11,Torch,0,0.01,0.1,,1,1,292,A torch sheds bright light in a 20-foot radius (and dim light to the next 20 feet). It can be used as an improvised weapon that deals 1d4 bludgeoning damage plus 1 fire damage.
|
||||
12,Soap,0,0.02,0,,1,1,288,
|
||||
13,Ladder,0,0.03,3,,2,1,288,
|
||||
14,Belt Pouch,0,0.04,0,,,1,289,A belt pouch holds up to four items of light Bulk.
|
||||
15,Flint and Steel,0,0.05,0,,2,1,290,"Flint and steel are useful in creating a fire if you have the time to catch a spark, though using them is typically too time-consuming to be practical during an encounter. Even in ideal conditions, using flint and steel to light a flame requires using at least 3 actions, and often significantly longer."
|
||||
16,Vial,0,0.05,0,,1,1,292,A simple glass vial holds up to 1 ounce of liquid.
|
||||
17,Waterskin,0,0.05,0.1,,1,1,292,"When it's full, a waterskin has 1 Bulk and contains roughly 1 day's worth of water for a Small or Medium creature."
|
||||
18,Signal Whistle,0,0.08,0,,1,1,291,"When sounded, a signal whistle can be heard clearly up to half a mile away across open terrain."
|
||||
19,Backpack,0,0.1,0,,,1,289,"A backpack holds up to 4 Bulk of items. If you're carrying or stowing the pack rather than wearing it on your back, its bulk is light instead of negligible."
|
||||
20,Bandolier,0,0.1,0,,,1,289,"A bandolier holds up to eight items of light Bulk within easy reach and is usually used for alchemical items or potions. If you are carrying or stowing a bandolier rather than wearing it around your chest, it has light Bulk instead of negligible. A bandolier can be dedicated to a full set of tools, such as healer's tools, allowing you to draw the tools as part of the action that requires them."
|
||||
21,Basic Crafter's Book,0,0.1,0.1,,2,1,289,This book contains the formulas (page 293) for Crafting the common items in this chapter.
|
||||
22,Clothing (Explorer's),0,0.1,0.1,,,1,287,"Explorer’s clothing is sturdy enough that it can be reinforced to protect you, even though it isn’t a suit of armor. It comes in many forms, though the most common sorts look like clerical vestments, monk’s garments, or wizard’s robes, as members of all three classes are likely to avoid wearing armor. For more information on explorer’s clothing, see pages 275–276."
|
||||
23,Clothing (Oridinary),0,0.1,0,,,1,287,"Ordinary clothing is functional with basic tailoring, such as peasant garb, monk’s robes, or work clothes."
|
||||
24,Disguise Kit (Replacement Cosmetics),0,0.1,0,,,1,290,"This small wooden box contains cosmetics, false facial hair, spirit gum, and a few simple wigs. You usually need a disguise kit to set up a disguise in order to Impersonate someone using the Deception skill. An elite disguise kit adds a +1 item bonus to relevant checks. If you’ve crafted a large number of disguises, you can replenish your cosmetics supply with replacement cosmetics suitable for the type of your disguise kit."
|
||||
25,Grappling Hook,0,0.1,0.1,,1,1,290,"You can throw a grappling hook with a rope tied to it to make a climb easier. To anchor a grappling hook, make an attack roll with the secret trait against a DC depending on the target, typically at least DC 20. On a success, your hook has a firm hold, but on a critical failure, the hook seems like it will hold but actually falls when you're partway up."
|
||||
26,Hammer,0,0.1,0.1,,1,1,288,
|
||||
27,Religious Symbol (Wooden),0,0.1,0.1,,1,1,291,"This piece of wood or silver is emblazoned with an image representing a deity. Some divine spellcasters, such as clerics, can use a religious symbol of their deity as a divine focus to use certain abilities and cast some spells. A religious symbol must be held in one hand to use it."
|
||||
28,Satchel,0,0.1,0,,2,1,291,"A satchel can hold up to 2 Bulk worth of items. If you are carrying or stowing a satchel rather than wearing it over your shoulder, it counts as light Bulk instead of negligible."
|
||||
29,Scroll Case,0,0.1,0,,2,1,291,"Scrolls, maps, and other rolled documents are stored in scroll cases for safe transport."
|
||||
30,Writing Set (Extra Ink and Paper),0,0.1,0,,,1,292,"Using a writing set, you can draft correspondence and scribe scrolls. A set includes stationery, including a variety of paper and parchment, as well as ink, a quill or inkpen, sealing wax, and a simple seal. If you’ve written a large amount, you can refill your kit with extra ink and paper."
|
||||
31,Lock (Poor),0,0.2,0,,2,1,290,Picking a poor lock requires two successful DC 15 Thievery checks.
|
||||
32,Merchant's Scale,0,0.2,0.1,,2,1,288,
|
||||
33,Saddlebags,0,0.2,0.1,,2,1,291,"Saddlebags come in a pair. Each can hold up to 3 Bulk of items. The Bulk value given is for saddlebags worn by a mount. If you are carrying or stowing saddlebags, they count as 1 Bulk instead of light Bulk."
|
||||
34,Caltrops,0,0.3,0.1,,1,1,289,"These four-pronged metal spikes can cause damage to a creature's feet. You can scatter caltrops in an empty square adjacent to you with an Interact action. The first creature that moves into that square must succeed at a DC 14 Acrobatics check or take 1d4 piercing damage and 1 persistent bleed damage. A creature taking persistent bleed damage from caltrops takes a 5-foot penalty to its Speed. Spending an Interact action to pluck the caltrops free reduces the DC to stop the bleeding. Once a creature takes damage from caltrops, enough caltrops are ruined that other creatures moving into the square are safe. Deployed caltrops can be salvaged and reused if no creatures took damage from them. Otherwise, enough caltrops are ruined that they can't be salvaged."
|
||||
35,Manacles (Poor),0,0.3,0,,2,1,290,"You can manacle someone who is willing or otherwise at your mercy as an exploration activity taking 10–30 seconds depending on the creature’s size and how many manacles you apply. A two-legged creature with its legs bound takes a –15-foot circumstance penalty to its Speeds, and a two-handed creature with its wrists bound has to succeed at a DC 5 flat check any time it uses a manipulate action or else that action fails. This DC may be higher depending on how tightly the manacles constrain the hands. A creature bound to a stationary object is immobilized. For creatures with more or fewer limbs, the GM determines what effect manacles have, if any. Freeing a creature from poor manacles requires two successful DC 17 Thievery checks."
|
||||
36,Thieves' Tools (Replacement Picks),0,0.3,0,,,1,291,"You need thieves’ tools to Pick Locks or Disable Devices (of some types) using the Thievery skill. If your thieves’ tools are broken, you can repair them by replacing the lock picks with replacement picks appropriate to your tools; this doesn’t require using the Repair action."
|
||||
37,Clothing (Winter),0,0.4,0.1,,,1,287,Winter clothing allows you to negate the damage from severe environmental cold and reduce the damage from extreme cold to that of severe cold.
|
||||
38,Rations,0,0.4,0.1,,1,1,288,
|
||||
39,Tool (Short),0,0.4,0.1,,1,1,292,"This entry is a catchall for basic hand tools that don’t have a specific adventuring purpose. A hoe, shovel, or sledgehammer is a long tool, and a hand drill, ice hook, or trowel is a short tool. A tool can usually be used as an improvised weapon, dealing 1d4 damage for a short tool or 1d6 for a long tool. The GM determines the damage type that’s appropriate or adjusts the damage if needed."
|
||||
40,Climbing Kit,0,0.5,1,,2,1,287,"This satchel includes 50 feet of rope, pulleys, a dozen pitons, a hammer, a grappling hook, and one set of crampons. Climbing kits allow you to attach yourself to the wall you're Climbing, moving half as quickly as usual (minimum 5 feet) but letting you attempt a DC 5 flat check whenever you critically fail to prevent a fall. A single kit has only enough materials for one climber; each climber needs their own kit."
|
||||
41,Climbing Kit (Extreme),3,40,1,,2,1,287,"This satchel includes 50 feet of rope, pulleys, a dozen pitons, a hammer, a grappling hook, and one set of crampons. Climbing kits allow you to attach yourself to the wall you’re Climbing, moving half as quickly as usual (minimum 5 feet) but letting you attempt a DC 5 flat check whenever you critically fail to prevent a fall. You gain a +1 item bonus to Athletics checks to Climb while using an extreme climbing kit. A single kit has only enough materials for one climber; each climber needs their own kit."
|
||||
42,Crowbar,0,0.5,0.1,,2,1,290,"When Forcing Open an object that doesn't have an easy grip, a crowbar makes it easier to gain the necessary leverage. Without a crowbar, prying something open takes a -2 item penalty to the Athletics check to Force Open (similar to using a shoddy item)."
|
||||
43,Material Component Pouch,0,0.5,0.1,,1,1,290,"This pouch contains material components for those spells that require them. Though the components are used up over time, you can refill spent components during your daily preparations."
|
||||
44,Rope,0,0.5,0.1,,2,1,288,
|
||||
45,Chest,0,0.6,2,,2,1,289,A wooden chest can hold up to 8 Bulk of items.
|
||||
46,Adventurer's Pack,0,0.7,2,,,1,289,"This item is the starter kit for an adventurer, containing the essential items for exploration and survival. The Bulk value is for the entire pack together, but see the descriptions of individual items as necessary. \n\n The pack contains the following items: backpack (containing the other goods), bedroll, two belt pouches, 10 pieces of chalk, flint and steel, 50 feet of rope, 2 weeks’ rations, soap, 5 torches, and a waterskin."
|
||||
47,Lantern (Hooded),0,0.7,0.1,,1,1,290,"A lantern sheds bright light and requires 1 pint of oil to function for 6 hours. \n\n A hooded lantern sheds light in a 30-foot radius (and dim light in the next 30 feet) and is equipped with shutters, which you can close to block the light. Closing or opening the shutters takes an Interact action."
|
||||
48,Fishing Tackle,0,0.8,1,,2,1,290,"This kit include a collapsible fishing pole, fishhooks, line, lures, and a fishing net."
|
||||
49,Musical Instrument (Handheld),0,0.8,1,,2,1,290,"Handheld instruments include bagpipes, a small set of chimes, small drums, fiddles and viols, flutes and recorders, small harps, lutes, trumpets, and similarly sized instruments. The GM might rule that an especially large handheld instrument (like a tuba) has greater Bulk."
|
||||
50,Tent (Pup),0,0.8,0.1,,2,1,288,
|
||||
51,Compass,0,1,0.1,,1,1,290,"A compass helps you Sense Direction or navigate, provided you’re in a location with uniform magnetic fields. Without a compass, you take a –2 item penalty to these checks (similar to using a shoddy item)."
|
||||
52,Cookware,0,1,2,,2,1,288,
|
||||
53,Formula Book (Blank),0,1,1,,1,1,290,"A formula book holds the formulas necessary to make items other than the common equipment from this chapter; alchemists typically get one for free. Each formula book can hold the formulas for up to 100 different items. Formulas can also appear on parchment sheets, tablets, and almost any other medium; there’s no need for you to copy them into a specific book as long as you can keep them on hand to reference them."
|
||||
54,Lantern (Bull's-Eye),0,1,1,,1,1,290,A lantern sheds bright light and requires 1 pint of oil to function for 6 hours. \n\n A bull’s-eye lantern emits its light in a 60-foot cone (and dim light in the next 60 feet).
|
||||
55,Mirror,0,1,0,,1,1,288,
|
||||
56,Religious Text,0,1,0.1,,1,1,291,"This manuscript contains scripture of a particular religion. Some divine spellcasters, such as clerics, can use a religious text as a divine focus to use certain abilities and cast some spells. A religious text must be held in one hand to use it."
|
||||
57,Spellbook (Blank),0,1,1,,1,1,291,"A spellbook holds the written knowledge necessary to learn and prepare various spells, a necessity for wizards (who typically get one for free) and a useful luxury for other spellcasters looking to learn additional spells. Each spellbook can hold up to 100 spells. The Price listed is for a blank spellbook."
|
||||
58,Tool (Long),0,1,1,,2,1,292,"This entry is a catchall for basic hand tools that don’t have a specific adventuring purpose. A hoe, shovel, or sledgehammer is a long tool, and a hand drill, ice hook, or trowel is a short tool. A tool can usually be used as an improvised weapon, dealing 1d4 damage for a short tool or 1d6 for a long tool. The GM determines the damage type that’s appropriate or adjusts the damage if needed."
|
||||
59,Writing Set,0,1,0.1,,2,1,292,"Using a writing set, you can draft correspondence and scribe scrolls. A set includes stationery, including a variety of paper and parchment, as well as ink, a quill or inkpen, sealing wax, and a simple seal. If you’ve written a large amount, you can refill your kit with extra ink and paper.Writing SetItem 0Source Core Rulebook pg. 292Price 1 gpHands 2; Bulk LWriting Set (Extra Ink and Paper)Item 0Source Core Rulebook pg. 292Price 1 sp"
|
||||
60,Clothing (Fine),0,2,0.1,,,1,287,"Fine clothing, suitable for a noble or royal, is made with expensive fabrics, precious metals, and intricate patterns."
|
||||
61,Disguise Kit,0,2,0.1,,2,1,290,"This small wooden box contains cosmetics, false facial hair, spirit gum, and a few simple wigs. You usually need a disguise kit to set up a disguise in order to Impersonate someone using the Deception skill. An elite disguise kit adds a +1 item bonus to relevant checks. If you’ve crafted a large number of disguises, you can replenish your cosmetics supply with replacement cosmetics suitable for the type of your disguise kit."
|
||||
62,Musical Instrument (Heavy),0,2,16,,2,1,290,"Handheld instruments include bagpipes, a small set of chimes, small drums, fiddles and viols, flutes and recorders, small harps, lutes, trumpets, and similarly sized instruments. The GM might rule that an especially large handheld instrument (like a tuba) has greater Bulk. Heavy instruments such as large drums, a full set of chimes, and keyboard instruments are less portable and generally need to be stationary while played."
|
||||
63,Religious Symbol (Silver),0,2,0.1,,1,1,291,"This piece of wood or silver is emblazoned with an image representing a deity. Some divine spellcasters, such as clerics, can use a religious symbol of their deity as a divine focus to use certain abilities and cast some spells. A religious symbol must be held in one hand to use it."
|
||||
64,Repair Kit,0,2,1,,2,1,291,"A repair kit allows you to perform simple repairs while traveling. It contains a portable anvil, tongs, woodworking tools, a whetstone, and oils for conditioning leather and wood. You can use a repair kit to Repair items using the Crafting skill."
|
||||
65,Hourglass,0,3,0.1,,1,1,288,
|
||||
66,Thieves' Tools,0,3,0.1,,2,1,291,"You need thieves’ tools to Pick Locks or Disable Devices (of some types) using the Thievery skill. If your thieves’ tools are broken, you can repair them by replacing the lock picks with replacement picks appropriate to your tools; this doesn’t require using the Repair action."
|
||||
67,Artisan's Tools,0,4,2,,2,1,289,"You need these tools to create items from raw materials with the Craft skill. Different sets are needed for different work, as determined by the GM; for example, blacksmith’s tools differ from woodworker’s tools."
|
||||
68,Chain,0,4,1,,2,1,288,
|
||||
69,Tack,0,4,1,,,1,291,"Tack includes all the gear required to outfit a riding animal, including a saddle, bit and bridle, and stirrups if necessary. Especially large or oddly shaped animals might require specialty saddles. These can be more expensive or hard to find, as determined by the GM. The Bulk value given is for tack worn by a creature. If carried, the Bulk increases to 2."
|
||||
70,Alchemist's Tools,0,5,2,,2,1,287,"These beakers and chemicals can be used to set up a mobile alchemical laboratory. When you carry the tools from place to place, you keep many of the components handy on your person, in pockets or bandoliers."
|
||||
71,Healer's Tools,0,5,1,,2,1,290,"This kit of bandages, herbs, and suturing tools is necessary for Medicine checks to Administer First Aid, Treat Disease, Treat Poison, or Treat Wounds. When you carry the tools from place to place, you keep many of the components handy on your person, in pockets or bandoliers."
|
||||
72,Snare Kit,0,5,2,,2,1,291,This kit contains tools and materials for creating snares. A snare kit allows you to Craft snares using the Crafting skill.
|
||||
73,Tent (Four-Person),0,5,1,,2,1,288,
|
||||
74,Spyglass,0,20,0.1,,2,1,291,A typical spyglass lets you see eight times farther than normal.
|
||||
75,Magnifying Glass,0,40,0,,1,1,290,"This quality handheld lens gives you a +1 item bonus to Perception checks to notice minute details of documents, fabric, and the like."
|
||||
76,Lock (Simple),1,2,0,,2,1,290,Picking a simple lock requires three successful DC 20 Thievery checks.
|
||||
77,Manacles (Simple),1,3,0,,2,1,290,"You can manacle someone who is willing or otherwise at your mercy as an exploration activity taking 10–30 seconds depending on the creature’s size and how many manacles you apply. A two-legged creature with its legs bound takes a –15-foot circumstance penalty to its Speeds, and a two-handed creature with its wrists bound has to succeed at a DC 5 flat check any time it uses a manipulate action or else that action fails. This DC may be higher depending on how tightly the manacles constrain the hands. A creature bound to a stationary object is immobilized. For creatures with more or fewer limbs, the GM determines what effect manacles have, if any. Freeing a creature from simple manacles requires three successes at DC 22."
|
||||
78,Tent (Pavilion),2,40,12,,2,1,288,
|
||||
79,Disguise Kit (Elite Cosmetics),3,0.5,0,,,1,290,"This small wooden box contains cosmetics, false facial hair, spirit gum, and a few simple wigs. You usually need a disguise kit to set up a disguise in order to Impersonate someone using the Deception skill. An elite disguise kit adds a +1 item bonus to relevant checks. If you’ve crafted a large number of disguises, you can replenish your cosmetics supply with replacement cosmetics suitable for the type of your disguise kit."
|
||||
80,Thieve's Tools (Infiltrator Picks),3,3,0,,,1,291,"You need thieves’ tools to Pick Locks or Disable Devices (of some types) using the Thievery skill. Infiltrator thieves’ tools add a +1 item bonus to checks to Pick Locks and Disable Devices. If your thieves’ tools are broken, you can repair them by replacing the lock picks with replacement picks appropriate to your tools; this doesn’t require using the Repair action."
|
||||
81,Scholarly Journal,3,6,0.1,Uncommon,1,1,291,"Scholarly journals are uncommon. Each scholarly journal is a folio on a very specific topic, such as vampires or the history of a single town or neighborhood of a city. If you spend 1 minute referencing an academic journal before attempting a skill check to Recall Knowledge about the subject, you gain a +1 item bonus to the check."
|
||||
82,Survey Map,3,10,0.1,Uncommon,1,1,291,"Maps are uncommon. Most maps you can find are simple and functional. A survey map details a single location in excellent detail. One of these maps gives you a +1 item bonus to Survival checks and any skill checks to Recall Knowledge, provided the checks are related to the location detailed on the map."
|
||||
83,Lock (Average),3,15,0,,2,1,290,Picking an average lock requires four successes at DC 25.
|
||||
84,Compass (Lensatic),3,20,0.1,,1,1,290,"A compass helps you Sense Direction or navigate, provided you’re in a location with uniform magnetic fields. Without a compass, you take a –2 item penalty to these checks (similar to using a shoddy item). A lensatic compass gives you a +1 item bonus to these checks."
|
||||
85,Crowbar (Levered),3,20,0.1,,2,1,290,"When Forcing Open an object that doesn’t have an easy grip, a crowbar makes it easier to gain the necessary leverage. Without a crowbar, prying something open takes a –2 item penalty to the Athletics check to Force Open (similar to using a shoddy item). A levered crowbar grants you a +1 item bonus to Athletics checks to Force Open anything that can be pried open. "
|
||||
86,Fishing Tackle (Professional),3,20,1,,2,1,290,"This kit include a collapsible fishing pole, fishhooks, line, lures, and a fishing net. Professional fishing tackle grants a +1 item bonus to checks to fish."
|
||||
87,Manacles (Average),3,20,0,,2,1,290,"You can manacle someone who is willing or otherwise at your mercy as an exploration activity taking 10–30 seconds depending on the creature’s size and how many manacles you apply. A two-legged creature with its legs bound takes a –15-foot circumstance penalty to its Speeds, and a two-handed creature with its wrists bound has to succeed at a DC 5 flat check any time it uses a manipulate action or else that action fails. This DC may be higher depending on how tightly the manacles constrain the hands. A creature bound to a stationary object is immobilized. For creatures with more or fewer limbs, the GM determines what effect manacles have, if any. Freeing a creature from average manacles require four successes at DC 27."
|
||||
88,Repair Kit (Superb),3,25,1,,2,1,291,"A repair kit allows you to perform simple repairs while traveling. It contains a portable anvil, tongs, woodworking tools, a whetstone, and oils for conditioning leather and wood. You can use a repair kit to Repair items using the Crafting skill. A superb repair kit gives you a +1 item bonus to the check."
|
||||
89,Scholarly Journal (Compendium),3,30,0.1,Uncommon,1,1,291,"Scholarly journals are uncommon. Each scholarly journal is a folio on a very specific topic, such as vampires or the history of a single town or neighborhood of a city. If you spend 1 minute referencing an academic journal before attempting a skill check to Recall Knowledge about the subject, you gain a +1 item bonus to the check. A compendium of journals costs five times as much as a single journal and requires both hands to use; each compendium contains several journals and grants its bonus on a broader topic, such as all undead or a whole city. The GM determines what scholarly journals are available in any location."
|
||||
90,Disguise Kit (Elite),3,40,0.1,,,1,290,"This small wooden box contains cosmetics, false facial hair, spirit gum, and a few simple wigs. You usually need a disguise kit to set up a disguise in order to Impersonate someone using the Deception skill. An elite disguise kit adds a +1 item bonus to relevant checks. If you’ve crafted a large number of disguises, you can replenish your cosmetics supply with replacement cosmetics suitable for the type of your disguise kit."
|
||||
91,Artisan's Tools (Sterling),3,50,2,,2,1,287,"You need these tools to create items from raw materials with the Craft skill. Sterling artisan’s tools give you a +1 item bonus to the check. Different sets are needed for different work, as determined by the GM; for example, blacksmith’s tools differ from woodworker’s tools."
|
||||
92,Healer's Tools (Expanded),3,50,1,,2,1,290,"This kit of bandages, herbs, and suturing tools is necessary for Medicine checks to Administer First Aid, Treat Disease, Treat Poison, or Treat Wounds. Expanded healer’s tools provide a +1 item bonus to such checks. When you carry the tools from place to place, you keep many of the components handy on your person, in pockets or bandoliers."
|
||||
93,Musical Instrument (Virtuoso Handheld),3,50,1,,2,1,290,"Handheld instruments include bagpipes, a small set of chimes, small drums, fiddles and viols, flutes and recorders, small harps, lutes, trumpets, and similarly sized instruments. The GM might rule that an especially large handheld instrument (like a tuba) has greater Bulk. \n\n A virtuoso instrument gives a +1 item bonus to Performance checks using that instrument."
|
||||
94,Survey Map (Atlas),3,50,0.1,Uncommon,2,1,291,"Maps are uncommon. Most maps you can find are simple and functional. A survey map details a single location in excellent detail. One of these maps gives you a +1 item bonus to Survival checks and any skill checks to Recall Knowledge, provided the checks are related to the location detailed on the map. Maps sometimes come in atlases, containing a number of maps of the same quality, often on similar topics. An atlas costs five times as much as a single map and requires both hands to use. The GM determines what maps are available in any location."
|
||||
95,Thieve's Tools (Infiltrator),3,50,0.1,,2,1,291,"You need thieves’ tools to Pick Locks or Disable Devices (of some types) using the Thievery skill. Infiltrator thieves’ tools add a +1 item bonus to checks to Pick Locks and Disable Devices. If your thieves’ tools are broken, you can repair them by replacing the lock picks with replacement picks appropriate to your tools; this doesn’t require using the Repair action."
|
||||
96,Alchemist's Tools (Expanded),3,55,2,,2,1,287,"These beakers and chemicals can be used to set up a mobile alchemical laboratory. Expanded alchemist’s tools give a +1 item bonus to Crafting checks to create alchemical items. When you carry the tools from place to place, you keep many of the components handy on your person, in pockets or bandoliers."
|
||||
97,Clothing (High-Fashion Fine),3,55,0.1,,,1,287,"Fine clothing, suitable for a noble or royal, is made with expensive fabrics, precious metals, and intricate patterns. You gain a +1 item bonus to checks to Make an Impression on nobility or other upper-class folk while wearing high-fashion fine clothing."
|
||||
98,Snare Kit (Specialist),3,55,2,,2,1,291,This kit contains tools and materials for creating snares. A snare kit allows you to Craft snares using the Crafting skill. A specialist snare kit gives you a +1 item bonus to the check.
|
||||
99,Musical Instrument (Virtuoso Heavy),3,100,16,,2,1,290,"Handheld instruments include bagpipes, a small set of chimes, small drums, fiddles and viols, flutes and recorders, small harps, lutes, trumpets, and similarly sized instruments. The GM might rule that an especially large handheld instrument (like a tuba) has greater Bulk. Heavy instruments such as large drums, a full set of chimes, and keyboard instruments are less portable and generally need to be stationary while played. \n\n A virtuoso instrument gives a +1 item bonus to Performance checks using that instrument. "
|
||||
100,Spyglass (Fine),4,80,0.1,,2,1,291,A typical spyglass lets you see eight times farther than normal. A fine spyglass adds a +1 item bonus to Perception checks to notice details at a distance.
|
||||
101,Lock (Good),9,200,0,,2,1,290,Picking a good lock requires five successes at DC 30.
|
||||
102,Manacles (Good),9,250,0,,2,1,290,"You can manacle someone who is willing or otherwise at your mercy as an exploration activity taking 10–30 seconds depending on the creature’s size and how many manacles you apply. A two-legged creature with its legs bound takes a –15-foot circumstance penalty to its Speeds, and a two-handed creature with its wrists bound has to succeed at a DC 5 flat check any time it uses a manipulate action or else that action fails. This DC may be higher depending on how tightly the manacles constrain the hands. A creature bound to a stationary object is immobilized. For creatures with more or fewer limbs, the GM determines what effect manacles have, if any. Freeing a creature from good manacles require five successes at DC 32."
|
||||
103,Lock (Superior),17,4500,0,,2,1,290,Picking a superior lock six successes at DC 40.
|
||||
104,Manacles (Superior),17,5000,0,,2,1,290,"You can manacle someone who is willing or otherwise at your mercy as an exploration activity taking 10–30 seconds depending on the creature’s size and how many manacles you apply. A two-legged creature with its legs bound takes a –15-foot circumstance penalty to its Speeds, and a two-handed creature with its wrists bound has to succeed at a DC 5 flat check any time it uses a manipulate action or else that action fails. This DC may be higher depending on how tightly the manacles constrain the hands. A creature bound to a stationary object is immobilized. For creatures with more or fewer limbs, the GM determines what effect manacles have, if any. Freeing a creature from superior manacles require six successes at DC 42."
|
||||
|
@@ -0,0 +1,173 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Scrape data from aon2e and generate csvs to import in to sqlite"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Dependencies\n",
|
||||
"import pandas as pd\n",
|
||||
"from bs4 import BeautifulSoup as bs, Tag, NavigableString\n",
|
||||
"import os\n",
|
||||
"from splinter import Browser\n",
|
||||
"\n",
|
||||
"# Setting up Selenium\n",
|
||||
"chrome_driver = os.path.join('..', 'chromedriver.exe')\n",
|
||||
"executable_path = {'executable_path': chrome_driver}\n",
|
||||
"browser = Browser('chrome', **executable_path, headless=False)\n",
|
||||
"\n",
|
||||
"# Pandas config\n",
|
||||
"pd.set_option('display.max_columns', None)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def scrape_description(url, id_number):\n",
|
||||
"\n",
|
||||
" # Empty lists to store the scraped values\n",
|
||||
" name_list = []\n",
|
||||
" description_list = []\n",
|
||||
"\n",
|
||||
" print(f'Beginning Data Retrieval')\n",
|
||||
" print(f'------------------------')\n",
|
||||
"\n",
|
||||
" # Loop from 1 to the value in weapon_number\n",
|
||||
" for page in range(1, id_number+1):\n",
|
||||
"\n",
|
||||
" browser.visit(url + str(page))\n",
|
||||
" html = browser.html\n",
|
||||
" soup = bs(html, 'html.parser')\n",
|
||||
"\n",
|
||||
" # Select only the content section\n",
|
||||
" content = soup.find(id='ctl00_MainContent_DetailedOutput')\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # Store the name and description\n",
|
||||
" name = content.find('h1', class_='title')\n",
|
||||
" name.span.decompose()\n",
|
||||
" name = name.text\n",
|
||||
"\n",
|
||||
" except:\n",
|
||||
" name = f'name: {page}'\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" description = ''\n",
|
||||
" start = content.find('hr')\n",
|
||||
" for e in start.next_siblings:\n",
|
||||
" if isinstance(e, Tag):\n",
|
||||
" description = description + e.text.strip()\n",
|
||||
" elif isinstance(e, NavigableString):\n",
|
||||
" description = description + e\n",
|
||||
"\n",
|
||||
" except:\n",
|
||||
" description = f'name: {page}'\n",
|
||||
"\n",
|
||||
" print(f'{page} of {id_number} | {name}')\n",
|
||||
"\n",
|
||||
" # Append values to our empty lists\n",
|
||||
" name_list.append(name) \n",
|
||||
" description_list.append(description)\n",
|
||||
"\n",
|
||||
" print(f'------------------------')\n",
|
||||
" print(f'Data Retrieval Complete')\n",
|
||||
" \n",
|
||||
" # Create df with the scraped data\n",
|
||||
" data = {'Name': name_list, 'description': description_list}\n",
|
||||
" \n",
|
||||
" # Returns a data frame\n",
|
||||
" return pd.DataFrame(data)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# scrape the descriptions\n",
|
||||
"url_gear = 'https://2e.aonprd.com/Equipment.aspx?ID='\n",
|
||||
"number_gear = 65 #65 to scrape\n",
|
||||
"\n",
|
||||
"gear_description = scrape_description(url_gear, number_gear)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Directory of csv files which are taken from https://2e.aonprd.com/Weapons.aspx\n",
|
||||
"gear = pd.read_csv('gear.csv')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"gear_description"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"gear = gear.merge(gear_description, how='left', on='Name')\n",
|
||||
"gear['Level'].fillna(0, inplace=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"gear.to_csv('gear.csv')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# # Scrape data from aon2e and generate csvs to import in to sqlite
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Dependencies
|
||||
import pandas as pd
|
||||
from bs4 import BeautifulSoup as bs, Tag, NavigableString
|
||||
import os
|
||||
from splinter import Browser
|
||||
|
||||
# Setting up Selenium
|
||||
chrome_driver = os.path.join('..', 'chromedriver.exe')
|
||||
executable_path = {'executable_path': chrome_driver}
|
||||
browser = Browser('chrome', **executable_path, headless=False)
|
||||
|
||||
# Pandas config
|
||||
pd.set_option('display.max_columns', None)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
def scrape_description(url, id_number):
|
||||
|
||||
# Empty lists to store the scraped values
|
||||
name_list = []
|
||||
description_list = []
|
||||
|
||||
print(f'Beginning Data Retrieval')
|
||||
print(f'------------------------')
|
||||
|
||||
# Loop from 1 to the value in weapon_number
|
||||
for page in range(1, id_number+1):
|
||||
|
||||
browser.visit(url + str(page))
|
||||
html = browser.html
|
||||
soup = bs(html, 'html.parser')
|
||||
|
||||
# Select only the content section
|
||||
content = soup.find(id='ctl00_MainContent_DetailedOutput')
|
||||
|
||||
try:
|
||||
# Store the name and description
|
||||
name = content.find('h1', class_='title')
|
||||
name.span.decompose()
|
||||
name = name.text
|
||||
|
||||
except:
|
||||
name = f'name: {page}'
|
||||
|
||||
try:
|
||||
description = ''
|
||||
start = content.find('hr')
|
||||
for e in start.next_siblings:
|
||||
if isinstance(e, Tag):
|
||||
description = description + e.text.strip()
|
||||
elif isinstance(e, NavigableString):
|
||||
description = description + e
|
||||
|
||||
except:
|
||||
description = f'name: {page}'
|
||||
|
||||
print(f'{page} of {id_number} | {name}')
|
||||
|
||||
# Append values to our empty lists
|
||||
name_list.append(name)
|
||||
description_list.append(description)
|
||||
|
||||
print(f'------------------------')
|
||||
print(f'Data Retrieval Complete')
|
||||
|
||||
# Create df with the scraped data
|
||||
data = {'Name': name_list, 'description': description_list}
|
||||
|
||||
# Returns a data frame
|
||||
return pd.DataFrame(data)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# scrape the descriptions
|
||||
url_gear = 'https://2e.aonprd.com/Equipment.aspx?ID='
|
||||
number_gear = 65 #65 to scrape
|
||||
|
||||
gear_description = scrape_description(url_gear, number_gear)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Directory of csv files which are taken from https://2e.aonprd.com/Weapons.aspx
|
||||
gear = pd.read_csv('gear.csv')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
gear_description
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
gear = gear.merge(gear_description, how='left', on='Name')
|
||||
gear['Level'].fillna(0, inplace=True)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
gear.to_csv('gear.csv')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
# This directory scrapes the weapons from the [Archives of Nethys](https://2e.aonprd.com/Weapons.aspx)
|
||||
|
||||
## Steps to scrape the weapons
|
||||
1. Install the requirements from [the previous readme](../README.md)
|
||||
2. Generate .csv files from copy pasting the tables from [here](https://2e.aonprd.com/Weapons.aspx) and save them in this directory
|
||||
3. Set the number_of_weapons variable to the number of weapons in the database it is currently 83
|
||||
4. Run the [python file](scrape.py) or [Jupyter Notebook](scrape.ipynb)
|
||||
@@ -0,0 +1,5 @@
|
||||
ammunition_id,name,price_gp,amount,bulk,group,source_id,source_pg,description
|
||||
1,Arrows,0.1,10,L,Bow,1,282,These projectiles are the ammunition for bows. The shaft of an arrow is made of wood. It is stabilized in flight by fletching at one end and bears a metal head on the other.
|
||||
2,Blowgun Darts,0.05,10,L,Dart,1,281,"These thin, light darts are typically made of hardwood and stabilized with fletching of down or fur. They are often hollow so they can be used to deliver poison."
|
||||
3,Bolts,0.1,10,L,Bow,1,281,"Shorter than traditional arrows but similar in construction, bolts are the ammunition used by crossbows."
|
||||
4,Sling Bullets,0.01,10,L,Sling,1,282,"These are small metal balls, typically either iron or lead, designed to be used as ammunition in slings."
|
||||
|
@@ -0,0 +1,68 @@
|
||||
weapon_id,name,category,price_gp,dice_size,damage_type,bulk,hands,group,traits,sources_id,sources_pg,description
|
||||
0,Fist,Unarmed,0,4,B,0,1,Brawling,"Agile, Finesse, Nonlethal, Unarmed",1,280,Nethys Note: no description was provided for this item
|
||||
1,Clan Dagger,Simple,2,4,P,0.1,1,Knife,"Agile, Dwarf, Parry, Uncommon, Versatile B",1,280,"This broad dagger is carried by dwarves as a weapon, tool, and designation of clan. Losing or having to surrender a clan dagger is considered a mark of embarrassment to most dwarves."
|
||||
2,Club,Simple,0,6,B,1,1,Club,"Thrown 10 ft., Thrown",1,280,This is a piece of stout wood shaped or repurposed to bludgeon an enemy. Clubs can be intricately carved pieces of martial art or as simple as a tree branch or piece of wood.
|
||||
3,Dagger,Simple,0.2,4,P,0.1,1,Knife,"Agile, Finesse, Thrown 10 ft., Versatile S, Thrown, Versatile",1,280,"This small, bladed weapon is held in one hand and used to stab a creature in close combat. It can also be thrown."
|
||||
4,Gauntlet,Simple,0.2,4,B,0.1,1,Brawling,"Agile, Free-Hand",1,280,"A pair of these metal gloves comes with full plate, half plate, and splint armor; they can also be purchased separately and worn with other types of armor. They not only protect your hands but also transform your hands into lethal weapons."
|
||||
5,Katar,Simple,0.3,4,P,0.1,1,Knife,"Agile, Deadly d6, Monk, Uncommon, Deadly",1,280,"Also known as punching daggers, katars are characterized by their H-shaped hand grip that allows the blade to jut out from the knuckles."
|
||||
6,Light Mace,Simple,0.4,4,B,0.1,1,Club,"Agile, Finesse, Shove",1,280,"A light mace has a short wooden or metal shaft ending with a dense metal head. Used much like a club, it delivers heavy bludgeoning blows, but with extra power derived from the head’s metal ridges or spikes."
|
||||
7,Longspear,Simple,0.5,8,P,2,2,Spear,Reach,1,280,"This very long spear, sometimes called a pike, is purely for thrusting rather than throwing. Used by many soldiers and city watch for crowd control and defense against charging enemies, it must be wielded with two hands."
|
||||
8,Mace,Simple,1,6,B,1,1,Club,Shove,1,280,"With a stout haft and a heavy metal head, a mace is sturdy and allows its wielder to deliver powerful blows and dent armor."
|
||||
9,Morningstar,Simple,1,6,B,1,1,Club,"Versatile P, Versatile",1,280,This weapon has a short shaft ending in a metal ball studded with spikes.
|
||||
10,Sickle,Simple,0.2,4,S,0.1,1,Knife,"Agile, Finesse, Trip",1,280,"Originally a farming tool used for reaping grain, this one-handed weapon has a short wooden handle ending in a curved blade, sometimes sharpened on both sides."
|
||||
11,Spear,Simple,0.1,6,P,1,1,Spear,"Thrown 20 ft., Thrown",1,280,"A long metal shaft ending with a metal spike, a spear can be used one-handed as a melee weapon and can be thrown."
|
||||
12,Spiked Gauntlet,Simple,0.3,4,P,0.1,1,Brawling,"Agile, Free-Hand",1,280,"Providing the same defensive function as a standard gauntlet, this version has a group of spikes protruding from the knuckles to deliver piercing damage with a punch."
|
||||
13,Staff,Simple,0,4,B,1,1,Club,"Two-Hand d8, Two-Hand",1,280,This long piece of wood can aid in walking and deliver a mighty blow.
|
||||
14,Bastard Sword,Martial,4,8,S,1,1,Sword,"Two-Hand d12, Two-Hand",1,280,"This broad-bladed sword, sometimes called the hand‑and‑a‑half sword, has a longer grip so it can be held in one hand or used with two hands to provide extra piercing or slashing power."
|
||||
15,Battle Axe,Martial,0.1,8,S,1,1,Axe,Sweep,1,280,"These axes are designed explicitly as weapons, rather than tools. They typically weigh less, with a shaft reinforced with metal bands or bolts, and have a sharper blade, making them ideal for chopping limbs rather than wood."
|
||||
16,Bo Staff,Martial,0.2,8,B,2,2,Club,"Monk, Parry, Reach, Trip",1,280,This strong but slender staff is tapered at the ends and well balanced. It’s designed to be an offensive and defensive weapon.
|
||||
17,Dogslicer,Martial,0.1,6,S,0.1,1,Sword,"Agile, Backstabber, Finesse, Goblin, Uncommon",1,280,"This short, curved, and crude makeshift blade often has holes drilled into it to reduce its weight. It’s a favored weapon of goblins."
|
||||
18,Elven Curve Blade,Martial,4,8,S,2,2,Sword,"Elf, Finesse, Forceful",1,281,"Essentially a longer version of the scimitar, this traditional elven weapon has a thinner blade than its cousin."
|
||||
19,Falchion,Martial,3,10,S,2,2,Sword,"Forceful, Sweep",1,280,"This weapon is a heavier, two-handed version of the curved-bladed scimitar. It is weighted toward the blade’s end, making it a powerful slashing weapon."
|
||||
20,Filcher's Fork,Martial,1,4,P,0.1,1,Spear,"Agile, Backstabber, Deadly d6, Finesse, Halfling, Thrown 20 ft., Uncommon, Deadly, Thrown",1,281,"This halfling weapon looks like a long, two-pronged fork and is used as both a weapon and a cooking implement."
|
||||
21,Flail,Martial,1,6,B,1,1,Flail,"Disarm, Sweep, Trip",1,280,"This weapon consists of a wooden handle attached to a spiked ball or cylinder by a chain, rope, or strap of leather."
|
||||
22,Glaive,Martial,2,8,S,2,2,Polearm,"Deadly d8, Forceful, Reach, Deadly",1,280,"This polearm consists of a long, single-edged blade on the end of a 7-foot pole. It is extremely effective at delivering lethal cuts at a distance."
|
||||
23,Gnome Hooked Hammer,Martial,2,6,B,1,1,Hammer,"Gnome, Trip, Two-Hand d10, Uncommon, Versatile P, Versatile, Two-Hand",1,281,This gnome tool and weapon features a hammer at one end and a curved pick on the other. It’s such a strange and awkward weapon that others think the gnomes are slightly erratic for using it.
|
||||
24,Greataxe,Martial,1,12,S,2,2,Axe,Sweep,1,280,"This large battle axe is too heavy to wield with only one hand. Many greataxes incorporate two blades, and they are often “bearded,” having a hook at the bottom to increase the strength of their chopping power."
|
||||
25,Greatclub,Martial,1,10,B,2,2,Club,"Backswing, Shove",1,280,"While many greatclubs are intricately carved, others are little more than a sturdy tree branch. These massive clubs are too heavy to wield with only one hand."
|
||||
26,Greatpick,Martial,2,10,P,2,2,Pick,"Fatal d12, Fatal",1,280,This pick has a longer handle and broader head than a regular pick. It is too heavy to wield in one hand.
|
||||
27,Greatsword,Martial,2,12,S,2,2,Sword,"Versatile P, Versatile",1,280,This immense two-handed sword is nearly as tall as its wielder. Its lower blade is often somewhat dulled to allow it to be gripped for extra leverage in close-quarter fights.
|
||||
28,Guisarme,Martial,2,10,S,2,2,Polearm,"Reach, Trip",1,280,"This polearm bears a long, often one‑sided, curved blade with a hook protruding from the blunt side of the blade, which can allow its wielder to trip opponents at a distance. Its shaft is usually 8 feet long."
|
||||
29,Halberd,Martial,2,10,P,2,2,Polearm,"Reach, Versatile S, Versatile",1,280,"This polearm has a relatively short, 5-foot shaft. The business end is a long spike with an axe blade attached."
|
||||
30,Hatchet,Martial,0.4,6,S,0.1,1,Axe,"Agile, Sweep, Thrown 10 ft., Thrown",1,280,This small axe can be used in close combat or thrown.
|
||||
31,Horsechopper,Martial,0.9,8,S,2,2,Polearm,"Goblin, Reach, Trip, Uncommon, Versatile P, Versatile",1,281,"Created by goblins to battle horses, this weapon is essentially a long shaft ending in a blade with a large hook."
|
||||
32,Kama,Martial,1,6,S,0.1,1,Knife,"Agile, Monk, Trip, Uncommon",1,281,"Similar to a sickle and used in some regions to reap grain, a kama has a short, slightly curved blade and a wooden handle."
|
||||
33,Katana,Martial,2,6,S,1,1,Sword,"Deadly d8, Two-Hand d10, Uncommon, Versatile P, Deadly, Two-Hand, Versatile",1,281,"A katana is a curved, single-edged sword known for its wickedly sharped blade."
|
||||
34,Kukri,Martial,0.6,6,S,0.1,1,Knife,"Agile, Finesse, Trip, Uncommon",1,281,The blade of this foot-long knife curves inward and lacks a cross guard at the hilt.
|
||||
35,Lance,Martial,1,8,P,2,2,Spear,"Deadly d8, Jousting d6, Reach, Deadly, Jousting",1,280,This spear-like weapon is used by a mounted creature to deal a great deal of damage.
|
||||
36,Light Hammer,Martial,0.3,6,B,0.1,1,Hammer,"Agile, Thrown 20 ft., Thrown",1,281,"This smaller version of the warhammer has a wooden or metal shaft ending in a metal head. Unlike its heavier cousin, it is light enough to throw."
|
||||
37,Light Pick,Martial,0.4,4,P,0.1,1,Pick,"Agile, Fatal d8, Fatal",1,281,A light pick is a modified mining implement with a wooden shaft ending in a pick head crafted more to pierce armor and flesh than chip rocks.
|
||||
38,Longsword,Martial,1,8,S,1,1,Sword,"Versatile P, Versatile",1,281,Longswords can be one-edged or two‑edged swords. Their blades are heavy and they’re between 3 and 4 feet in length.
|
||||
39,Main-gauche,Martial,0.5,4,P,0.1,1,Knife,"Agile, Disarm, Finesse, Parry, Versatile S, Versatile",1,281,This parrying dagger features a robust guard to protect the wielder’s hand.
|
||||
40,Maul,Martial,3,12,B,2,2,Hammer,Shove,1,281,Mauls are massive warhammers that must be swung with two hands.
|
||||
41,Nunchaku,Martial,0.2,6,B,0.1,1,Club,"Backswing, Disarm, Finesse, Monk, Uncommon",1,281,The nunchaku is constructed of two wooden or metal bars connected by a short length of rope or chain.
|
||||
42,Orc Knuckle Dagger,Martial,0.7,6,P,0.1,1,Knife,"Agile, Disarm, Orc, Uncommon",1,281,"This stout, metal blade of orc design has a horizontal basket hilt with blades jutting from each end, or sometimes one blade like that of a katar."
|
||||
43,Pick,Martial,0.7,6,P,1,1,Pick,"Fatal d10, Fatal",1,281,"A pick designed solely for combat has a sturdy wooden shaft and a heavy, pointed head to deliver devastating blows."
|
||||
44,Ranseur,Martial,2,10,P,2,2,Polearm,"Disarm, Reach",1,281,This polearm is a long trident with a central prong that’s longer than the other two.
|
||||
45,Rapier,Martial,2,6,P,1,1,Sword,"Deadly d8, Disarm, Finesse, Deadly",1,281,The rapier is a long and thin piercing blade with a basket hilt. It is prized among many as a dueling weapon.
|
||||
46,Sai,Martial,0.6,4,P,0.1,1,Knife,"Agile, Disarm, Finesse, Monk, Uncommon, Versatile B, Versatile",1,281,This piercing dagger is a metal spike flanked by a pair of prongs that can be used to trap an enemy’s weapon.
|
||||
47,Sap,Martial,0.1,6,B,0.1,1,Club,"Agile, Nonlethal",1,281,"A sap has a soft wrapping around a dense core, typically a leather sheath around a lead rod. Its head is wider than its grip to disperse the force of a blow, as the weapon’s purpose is to knock out its victim rather than to draw blood."
|
||||
48,Scimitar,Martial,1,6,S,1,1,Sword,"Forceful, Sweep",1,281,This one-handed curved blade is sharp on one side.
|
||||
49,Scythe,Martial,2,10,S,2,2,Polearm,"Deadly d10, Trip, Deadly",1,281,"Derived from a farming tool used to mow down long grains and cereals, this weapon has a long wooden shaft with protruding handles, capped with a curved blade set at a right angle."
|
||||
50,Shield Bash,Martial,0,4,B,0,1,Shield,,1,281,"A shield bash is not actually a weapon, but a maneuver in which you thrust or swing your shield to hit your foe with an impromptu attack."
|
||||
51,Shield Boss,Martial,0.5,6,B,0,1,Shield,Attached to Shield,1,281,"Typically a round, convex, or conical piece of thick metal attached to the center of a shield, a shield boss increases the bludgeoning damage of a shield bash."
|
||||
52,Shield Spikes,Martial,0.5,6,P,0,1,Shield,Attached to Shield,1,281,These metal spikes are strategically placed on the defensive side of the shield to deal piercing damage with a shield bash.
|
||||
53,Shortsword,Martial,0.9,6,P,0.1,1,Sword,"Agile, Finesse, Versatile S, Versatile",1,281,"These blades come in a variety of shapes and styles, but they are typically 2 feet long."
|
||||
54,Spiked Chain,Martial,3,8,S,1,2,Flail,"Disarm, Finesse, Trip, Uncommon",1,281,This 4‑foot‑long length of chain is covered with barbs and has spikes on one or both ends. Some feature metal hoops used as handgrips.
|
||||
55,Starknife,Martial,2,4,P,0.1,1,Knife,"Agile, Deadly d6, Finesse, Thrown 20 ft., Versatile S, Deadly, Thrown, Versatile",1,281,"From a central metal ring, four tapering metal blades extend like points on a compass rose. When gripping a starknife from the center, the wielder can use it as a melee weapon. It can also be thrown short distances."
|
||||
56,Temple Sword,Martial,2,8,S,1,1,Sword,"Monk, Trip, Uncommon",1,281,"This heavy blade is favored by guardians of religious sites. It has a distinctive, crescent-shaped blade that seems to be a mix of a sickle and sword. It often has holes drilled into the blade or the pommel so that bells or other holy trinkets can be affixed to the weapon as an aid for prayer or mediation."
|
||||
57,Trident,Martial,1,8,P,1,1,Spear,"Thrown 20 ft., Thrown",1,281,"This three-pronged, spear-like weapon typically has a 4-foot shaft. Like a spear, it can be wielded with one hand or thrown."
|
||||
58,War Flair,Martial,2,10,B,2,2,Flail,"Disarm, Sweep, Trip",1,281,
|
||||
59,Warhammer,Martial,1,8,B,1,1,Hammer,Shove,1,281,"This weapon has a wooden shaft ending in a large, heavy metal head. The head of the hammer might be single-sided or double-sided, but it’s always capable of delivering powerful bludgeoning blows."
|
||||
60,Whip,Martial,0.1,4,S,1,1,Flail,"Disarm, Finesse, Nonlethal, Reach, Trip",1,281,"This long strand of thick leather, often braided, delivers a painful but nonlethal slash at a distance, usually accompanied by a distinctive cracking sound."
|
||||
61,Aklys,Advanced,5,6,B,1,1,Club,"Ranged Trip, Tethered, Thrown 20 feet, Trip, Uncommon, Thrown",2,85,
|
||||
62,Dwarven War Axe,Advanced,3,8,S,2,1,Axe,"Dwarf, Sweep, Two-Hand d12, Two-Hand",1,281,"This favored weapon of the dwarves has a large, ornate head mounted on a thick handle. This powerful axe can be wielded with one hand or two."
|
||||
63,Gnome Flickmace,Advanced,3,8,B,2,1,Flail,"Gnome, Reach, Uncommon",1,281,"More a flail than a mace, this weapon has a short handle attached to a length of chain with a ball at the end. The ball is propelled to its reach with the flick of the wrist, the momentum of which brings the ball back to the wielder after the strike."
|
||||
64,Orc Necksplitter,Advanced,2,8,S,1,1,Axe,"Forceful, Orc, Sweep, Uncommon",1,281,This single-bladed bearded axe has a jagged blade that’s perfect for separating bone from tendon and cartilage.
|
||||
65,Sawtooth Saber,Advanced,5,8,S,0.1,1,Sword,"Agile, Finesse, Twin, Uncommon",1,281,"The signature weapon of the Red Mantis assassins, this curved blade is serrated like a saw, hence the name."
|
||||
66,Ogre Hook,Advanced,1,10,P,2,2,Pick,"Deadly 1d10, Trip, Uncommon, Deadly",2,253,"Ogres are known for using immense, curved picks called ogre hooks."
|
||||
|
@@ -0,0 +1,15 @@
|
||||
weapon_id,name,category,price_gp,dice_size,damage_type,range,reload,bulk,hands,group,traits,sources_id,sources_pg,description
|
||||
67,Blowgun,Simple,0.1,1,P,20,1,0.1,1,Dart,"Agile, Nonlethal",1,281,"This long, narrow tube is used for shooting blowgun darts, using only the power of a forcefully exhaled breath."
|
||||
68,Crossbow,Simple,3,8,P,120,1,1,2,Bow,,1,281,"This ranged weapon has a bow-like assembly mounted on a handled frame called a tiller. The tiller has a mechanism to lock the bowstring in place, attached to a trigger mechanism that releases the tension and launches a bolt."
|
||||
69,Dart,Simple,0.01,4,P,20,,0.1,1,Dart,"Agile, Thrown",1,281,This thrown weapon is larger than an arrow but shorter than a javelin. It typically has a short shaft of wood ending in a metal tip and is sometimes stabilized by feathers or fur.
|
||||
70,Hand Crossbow,Simple,3,6,P,60,1,0.1,1,Bow,,1,281,"Sometimes referred to as an alley bow by rogues or ruffians, this small crossbow fires small bolts that are sometimes used to deliver poison to the target. It's small enough to be shot one-handed, but it still requires two hands to load."
|
||||
71,Heavy Crossbow,Simple,4,10,P,120,2,2,2,Bow,,1,281,"This large crossbow is harder to load and more substantial than a regular crossbow, but it packs a greater punch."
|
||||
72,Javelin,Simple,0.1,6,P,30,,0.1,1,Dart,Thrown,1,282,This thin spear is well balanced for throwing but is not designed for melee use.
|
||||
73,Sling,Simple,0,6,B,50,1,0.1,1,Sling,Propulsive,1,282,"Little more than a leather cup attached to a pair of straps, a sling can be used to fling smooth stones or sling bullets at a range."
|
||||
74,Alchemical Bomb,Martial,,,,20,,0.1,1,Bomb,,1,282,"These bombs come in a variety of types and levels of power, but no matter the variety, you throw the bomb at the target and it explodes, unleashing its alchemical blast."
|
||||
75,Composite Longbow,Martial,20,8,P,100,0,2,1+,Bow,"Deadly d10, Propulsive, Volley 30 ft., Deadly, Volley",1,282,"This projectile weapon is made from horn, wood, and sinew laminated together to increase the power of its pull and the force of its projectile. Like all longbows, its great size also increases the bow's range and power. You must use two hands to fire it, and it cannot be used while mounted. Any time an ability is specifically restricted to a longbow, such as Erastil's favored weapon, it also applies to composite longbows unless otherwise stated."
|
||||
76,Composite Shortbow,Martial,14,6,P,60,0,1,1+,Bow,"Deadly d10, Propulsive, Deadly",1,282,"This shortbow is made from horn, wood, and sinew laminated together to increase the power of its pull and the force of its projectile. Its compact size and power make it a favorite of mounted archers. Any time an ability is specifically restricted to a shortbow, it also applies to composite shortbows unless otherwise stated."
|
||||
77,Halfling Sling Staff,Martial,5,10,B,80,1,1,2,Sling,"Halfling, Propulsive, Uncommon",1,282,This staff ends in a Y-shaped split that cradles a sling. The length of the staff provides excellent leverage when used two‑handed to fling rocks or bullets from the sling.
|
||||
78,Longbow,Martial,6,8,P,100,0,2,1+,Bow,"Deadly d10, Volley 30 ft., Deadly, Volley",1,282,"This 5-foot-tall bow, usually made of a single piece of elm, hickory, or yew, has a powerful draw and is excellent at propelling arrows with great force and at an extreme distance. You must use two hands to fire a longbow, and it can't be used while mounted."
|
||||
79,Shortbow,Martial,3,6,P,60,0,1,1+,Bow,"Deadly d10, Deadly",1,282,This smaller bow is made of a single piece of wood and favored by skirmishers and cavalry.
|
||||
80,Shuriken,Martial,0.01,4,P,20,0,,1,Dart,"Agile, Monk, Thrown, Uncommon",1,282,"This throwing star is a small piece of flat metal with sharp edges, designed to be flung with a flick of the wrist."
|
||||
|
@@ -0,0 +1,162 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Scrape data from aon2e and generate csvs to import in to sqlite"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Dependencies\n",
|
||||
"import pandas as pd\n",
|
||||
"from bs4 import BeautifulSoup as bs\n",
|
||||
"import os\n",
|
||||
"from splinter import Browser\n",
|
||||
"\n",
|
||||
"# Setting up Selenium\n",
|
||||
"chrome_driver = os.path.join('..', 'chromedriver.exe')\n",
|
||||
"executable_path = {'executable_path': chrome_driver}\n",
|
||||
"browser = Browser('chrome', **executable_path, headless=False)\n",
|
||||
"\n",
|
||||
"# Pandas config\n",
|
||||
"pd.set_option('display.max_columns', None)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# url that contains all the links\n",
|
||||
"url_weapon = 'https://2e.aonprd.com/Weapons.aspx?ID='\n",
|
||||
"\n",
|
||||
"# Number of weapons\n",
|
||||
"number_of_weapons = 83\n",
|
||||
"\n",
|
||||
"# Empty lists to store the scraped values\n",
|
||||
"name_list = []\n",
|
||||
"description_list = []\n",
|
||||
"\n",
|
||||
"print(f'Beginning Data Retrieval')\n",
|
||||
"print(f'------------------------')\n",
|
||||
"\n",
|
||||
"# Loop from 1 to the value in weapon_number\n",
|
||||
"for weapon in range(1, number_of_weapons+1):\n",
|
||||
" \n",
|
||||
" url = url_weapon + str(weapon)\n",
|
||||
" browser.visit(url)\n",
|
||||
" html = browser.html\n",
|
||||
" soup = bs(html, 'html.parser')\n",
|
||||
"\n",
|
||||
" # Select only the content section\n",
|
||||
" content = soup.find(id='ctl00_MainContent_DetailedOutput')\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # Store the name and description\n",
|
||||
" name = content.find('a').text.strip()\n",
|
||||
" \n",
|
||||
" except:\n",
|
||||
" name = f'weapon: {weapon}'\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" description = content.find('hr').next.text.strip()\n",
|
||||
" \n",
|
||||
" except:\n",
|
||||
" description = content.find('hr').next.strip()\n",
|
||||
" \n",
|
||||
" print(f'Processing Weapon {weapon} of {number_of_weapons} | {name}')\n",
|
||||
"\n",
|
||||
" # Append values to our empty lists\n",
|
||||
" name_list.append(name) \n",
|
||||
" description_list.append(description)\n",
|
||||
"\n",
|
||||
"print(f'------------------------')\n",
|
||||
"print(f'Data Retrieval Complete')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Directory of csv files which are taken from https://2e.aonprd.com/Weapons.aspx\n",
|
||||
"melee = pd.read_csv('melee.csv')\n",
|
||||
"ranged = pd.read_csv('ranged.csv')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"data = {'Name': name_list, 'description': description_list}\n",
|
||||
"scrape = pd.DataFrame(data)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"melee = melee.merge(scrape, how='left', on='Name')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"ranged = ranged.merge(scrape, how='left', on='Name')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"melee.to_csv('melee.csv')\n",
|
||||
"ranged.to_csv('ranged.csv')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# # Scrape data from aon2e and generate csvs to import in to sqlite
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Dependencies
|
||||
import pandas as pd
|
||||
from bs4 import BeautifulSoup as bs
|
||||
import os
|
||||
from splinter import Browser
|
||||
|
||||
# Setting up Selenium
|
||||
chrome_driver = os.path.join('..', 'chromedriver.exe')
|
||||
executable_path = {'executable_path': chrome_driver}
|
||||
browser = Browser('chrome', **executable_path, headless=False)
|
||||
|
||||
# Pandas config
|
||||
pd.set_option('display.max_columns', None)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# url that contains all the links
|
||||
url_weapon = 'https://2e.aonprd.com/Weapons.aspx?ID='
|
||||
|
||||
# Number of weapons
|
||||
number_of_weapons = 83
|
||||
|
||||
# Empty lists to store the scraped values
|
||||
name_list = []
|
||||
description_list = []
|
||||
|
||||
print(f'Beginning Data Retrieval')
|
||||
print(f'------------------------')
|
||||
|
||||
# Loop from 1 to the value in weapon_number
|
||||
for weapon in range(1, number_of_weapons+1):
|
||||
|
||||
url = url_weapon + str(weapon)
|
||||
browser.visit(url)
|
||||
html = browser.html
|
||||
soup = bs(html, 'html.parser')
|
||||
|
||||
# Select only the content section
|
||||
content = soup.find(id='ctl00_MainContent_DetailedOutput')
|
||||
|
||||
try:
|
||||
# Store the name and description
|
||||
name = content.find('a').text.strip()
|
||||
|
||||
except:
|
||||
name = f'weapon: {weapon}'
|
||||
|
||||
try:
|
||||
description = content.find('hr').next.text.strip()
|
||||
|
||||
except:
|
||||
description = content.find('hr').next.strip()
|
||||
|
||||
print(f'Processing Weapon {weapon} of {number_of_weapons} | {name}')
|
||||
|
||||
# Append values to our empty lists
|
||||
name_list.append(name)
|
||||
description_list.append(description)
|
||||
|
||||
print(f'------------------------')
|
||||
print(f'Data Retrieval Complete')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
# Directory of csv files which are taken from https://2e.aonprd.com/Weapons.aspx
|
||||
melee = pd.read_csv('melee.csv')
|
||||
ranged = pd.read_csv('ranged.csv')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
data = {'Name': name_list, 'description': description_list}
|
||||
scrape = pd.DataFrame(data)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
melee = melee.merge(scrape, how='left', on='Name')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
ranged = ranged.merge(scrape, how='left', on='Name')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
melee.to_csv('melee.csv')
|
||||
ranged.to_csv('ranged.csv')
|
||||
|
||||
|
||||
# In[ ]:
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
import csv
|
||||
import sqlite3
|
||||
|
||||
def main():
|
||||
# load json into python
|
||||
print("loading melee.csv and ranged.csv")
|
||||
## read file into memory
|
||||
rows = []
|
||||
with open('melee.csv') as f:
|
||||
reader = csv.DictReader(f)
|
||||
# print(reader)
|
||||
for row in reader:
|
||||
rows.append(row)
|
||||
|
||||
ranged_rows = []
|
||||
with open('ranged.csv') as f:
|
||||
reader = csv.DictReader(f)
|
||||
# print(reader)
|
||||
for row in reader:
|
||||
ranged_rows.append(row)
|
||||
|
||||
|
||||
## Get database connection
|
||||
conn = sqlite3.connect('../../pf2.db')
|
||||
## Set pragmas to make it go FAST
|
||||
pragma1 = "PRAGMA synchronous=OFF;"
|
||||
pragma2 = "PRAGMA count_changes=OFF;"
|
||||
pragma3 = "PRAGMA journal_mode=MEMORY;"
|
||||
pragma4 = "PRAGMA temp_store=MEMORY;"
|
||||
pragma5 = "PRAGMA foreign_keys=ON;"
|
||||
conn.execute(pragma1)
|
||||
conn.execute(pragma2)
|
||||
conn.execute(pragma3)
|
||||
conn.execute(pragma4)
|
||||
conn.execute(pragma5)
|
||||
|
||||
# load in ids for traits from traits table so we only call this once
|
||||
# instead of every spell
|
||||
stmt = "SELECT trait_id, short_name FROM traits"
|
||||
c = conn.cursor()
|
||||
c.execute(stmt)
|
||||
traits = c.fetchall()
|
||||
# print("Printing traits")
|
||||
# print(traits)
|
||||
|
||||
for row in rows:
|
||||
insert_melee_weapon_basics(row, conn)
|
||||
insert_traits(row, conn, traits)
|
||||
insert_fks(row, conn)
|
||||
|
||||
for row in ranged_rows:
|
||||
insert_ranged_weapon_basics(row, conn)
|
||||
insert_traits(row, conn, traits)
|
||||
insert_fks(row, conn)
|
||||
|
||||
print("finished weapons")
|
||||
|
||||
def insert_fks(row, conn):
|
||||
# get category id
|
||||
rcat = row['category']
|
||||
res_id = None
|
||||
if rcat == "Unarmed":
|
||||
res_id = 1
|
||||
elif rcat == "Simple":
|
||||
res_id = 2
|
||||
elif rcat == "Martial":
|
||||
res_id = 3
|
||||
elif rcat == "Advanced":
|
||||
res_id = 4
|
||||
else:
|
||||
print("Something went wrong here in weapon category land.")
|
||||
|
||||
grp = row['group']
|
||||
grp_id = None
|
||||
if grp == 'Axe':
|
||||
grp_id = 1
|
||||
elif grp == 'Bomb':
|
||||
grp_id = 2
|
||||
elif grp == 'Bow':
|
||||
grp_id = 3
|
||||
elif grp == 'Brawling':
|
||||
grp_id = 4
|
||||
elif grp == 'Club':
|
||||
grp_id = 5
|
||||
elif grp == 'Dart':
|
||||
grp_id = 6
|
||||
elif grp == 'Flail':
|
||||
grp_id = 7
|
||||
elif grp == 'Hammer':
|
||||
grp_id = 8
|
||||
elif grp == 'Knife':
|
||||
grp_id = 9
|
||||
elif grp == 'Pick':
|
||||
grp_id = 10
|
||||
elif grp == 'Polearm':
|
||||
grp_id = 11
|
||||
elif grp == 'Shield':
|
||||
grp_id = 12
|
||||
elif grp == 'Sling':
|
||||
grp_id = 13
|
||||
elif grp == 'Spear':
|
||||
grp_id = 14
|
||||
elif grp == 'Sword':
|
||||
grp_id = 15
|
||||
else:
|
||||
print("Something went wrong here in weapon group land.")
|
||||
|
||||
dmg = row['damage_type']
|
||||
dmg_id = None
|
||||
if dmg == 'B':
|
||||
dmg_id = 1
|
||||
elif dmg == 'P':
|
||||
dmg_id = 2
|
||||
elif dmg == 'S':
|
||||
dmg_id = 3
|
||||
elif row['name'] == "Alchemical Bomb":
|
||||
dmg_id = None
|
||||
else:
|
||||
print("Something went wrong here in weapon damage type land.{}".format(row))
|
||||
|
||||
inp = (res_id, grp_id, dmg_id, row['weapon_id'])
|
||||
stmt = "UPDATE weapons SET weaponcategories_id=?, weapongroups_id=?, damagetypes_id=? WHERE weapons_id=?;"
|
||||
try:
|
||||
conn.execute(stmt, inp)
|
||||
except sqlite.Error as e:
|
||||
print("Error inserting weapon category information: {}".format(e))
|
||||
else:
|
||||
conn.commit()
|
||||
# print("Successfully inserted row")
|
||||
|
||||
def insert_traits (row, conn, traits):
|
||||
|
||||
# get list of traits from row
|
||||
tmp = row['traits']
|
||||
# exit this if no traits
|
||||
if tmp == '':
|
||||
return
|
||||
splits = tmp.split(", ")
|
||||
# print(tmp)
|
||||
# print(tmp.split(", "))
|
||||
|
||||
trait_ids = []
|
||||
# get trait IDs
|
||||
for i in traits:
|
||||
for j in splits:
|
||||
if i[1] == j:
|
||||
# print("Trait_id:{}\tname:{}".format(i[0],i[1]))
|
||||
trait_ids.append(i[0])
|
||||
# print(trait_ids)
|
||||
# all this mess removes non weapon traits that have duplicate names like
|
||||
# Monk Ancestry versus Monk Weapon trait
|
||||
if len(splits) != len(trait_ids):
|
||||
remove_me = []
|
||||
for i in trait_ids:
|
||||
if i >= 217 and i <= 251:
|
||||
continue
|
||||
elif i >= 255 and i <= 269:
|
||||
continue
|
||||
# 203 = uncommon trait
|
||||
elif i == 203:
|
||||
continue
|
||||
else:
|
||||
remove_me.append(i)
|
||||
for i in remove_me:
|
||||
trait_ids.remove(i)
|
||||
# print(trait_ids)
|
||||
|
||||
stmt = "INSERT INTO weapons_traits (weapons_id, trait_id) VALUES (?,?);"
|
||||
|
||||
for i in trait_ids:
|
||||
inp = (row['weapon_id'], i)
|
||||
try:
|
||||
conn.execute(stmt, inp)
|
||||
except sqlite.Error as e:
|
||||
print("Error inserting trait information: {}".format(e))
|
||||
else:
|
||||
conn.commit()
|
||||
# print("Successfully inserted row")
|
||||
|
||||
|
||||
|
||||
def insert_ranged_weapon_basics(row, conn):
|
||||
# print("Inserting: {}".format(row['name']))
|
||||
# insert everything that's not a lookup
|
||||
stmt = """
|
||||
INSERT INTO weapons (
|
||||
weapons_id,
|
||||
sources_id,
|
||||
sources_pages,
|
||||
price_gp,
|
||||
dice_size,
|
||||
bulk,
|
||||
hands,
|
||||
name,
|
||||
descr,
|
||||
range,
|
||||
reload
|
||||
)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?);
|
||||
"""
|
||||
r = row
|
||||
inp = (r['weapon_id'],r['sources_id'],r['sources_pg'],r['price_gp'],
|
||||
r['dice_size'],r['bulk'],r['hands'],r['name'],r['description'],
|
||||
r['range'],r['reload'])
|
||||
|
||||
try:
|
||||
conn.execute(stmt, inp)
|
||||
except sqlite.Error as e:
|
||||
print("Error inserting basic ranged row information: {}".format(e))
|
||||
else:
|
||||
conn.commit()
|
||||
# print("Successfully inserted row")
|
||||
|
||||
def insert_melee_weapon_basics(row, conn):
|
||||
# print("Inserting: {}".format(row['name']))
|
||||
# insert everything that's not a lookup
|
||||
stmt = """
|
||||
INSERT INTO weapons (
|
||||
weapons_id,
|
||||
sources_id,
|
||||
sources_pages,
|
||||
price_gp,
|
||||
dice_size,
|
||||
bulk,
|
||||
hands,
|
||||
name,
|
||||
descr)
|
||||
VALUES (?,?,?,?,?,?,?,?,?);
|
||||
"""
|
||||
r = row
|
||||
inp = (r['weapon_id'],r['sources_id'],r['sources_pg'],r['price_gp'],
|
||||
r['dice_size'],r['bulk'],r['hands'],r['name'],r['description'])
|
||||
|
||||
try:
|
||||
conn.execute(stmt, inp)
|
||||
except sqlite3.Error as e:
|
||||
print("Error inserting basic melee row information: {}".format(e))
|
||||
else:
|
||||
conn.commit()
|
||||
# print("Successfully inserted row")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,4 @@
|
||||
pandas==0.24.2
|
||||
splinter==0.11.0
|
||||
beautifulsoup4==4.8.0
|
||||
selenium==3.141.0
|
||||
@@ -0,0 +1,43 @@
|
||||
id,name,level,price,bulk,usage,cantrips,1,2,3,4,5,6,7,8,9,source_id,source_pg,description
|
||||
,Animal Staff,4,90,1,,,,,,,,,,,,,,
|
||||
,Animal Staff (Greater),8,460,1,,,,,,,,,,,,,,
|
||||
,Animal Staff (Major),12,1900,1,,,,,,,,,,,,,,
|
||||
,Mentalist's Staff,4,230,1,,,,,,,,,,,,,,
|
||||
,Mentalist's Staff (Greater),8,900,1,,,,,,,,,,,,,,
|
||||
,Mentalist's Staff (Major),12,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Abjuration,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Abjuration (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Abjuration (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Conjuration,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Conjuration (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Conjuration (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Divination,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Divination (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Divination (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Enchantment,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Enchantment (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Enchantment (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Evocation,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Evocation (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Evocation (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Fire,3,60,1,held in 1 hand,produce flame,burning hands,,,,,,,,,1,595,"This staff resembles a blackened and burned length of ashen wood. It smells faintly of soot and glows as if lit by embers. You can use an Interact action to touch the tip of this staff to a torch, tinder, or a flammable substance to ignite a flame. \n\n Activate Cast a Spell; Effect You expend a number of charges from the staff to cast a spell from its list. Craft Requirements Supply one casting of all listed levels of all listed spells."
|
||||
,Staff of Fire (Greater),8,450,1,held in 1 hand,produce flame,burning hands,"burning hands, flaming sphere","flaming sphere, fireball",,,,,,,1,595,"This staff resembles a blackened and burned length of ashen wood. It smells faintly of soot and glows as if lit by embers. You can use an Interact action to touch the tip of this staff to a torch, tinder, or a flammable substance to ignite a flame. \n\n Activate Cast a Spell; Effect You expend a number of charges from the staff to cast a spell from its list. Craft Requirements Supply one casting of all listed levels of all listed spells."
|
||||
,Staff of Fire (Major),12,1800,1,held in 1 hand,produce flame,burning hands,"burning hands, flaming sphere","flaming sphere, fireball","fire shield, fireball, wall of fire","fireball, wall of fire",,,,,1,595,"This staff resembles a blackened and burned length of ashen wood. It smells faintly of soot and glows as if lit by embers. You can use an Interact action to touch the tip of this staff to a torch, tinder, or a flammable substance to ignite a flame. \n\n Activate Cast a Spell; Effect You expend a number of charges from the staff to cast a spell from its list. Craft Requirements Supply one casting of all listed levels of all listed spells."
|
||||
,Staff of Healing,4,90,1,,,,,,,,,,,,,,
|
||||
,Staff of Healing (Greater),8,470,1,,,,,,,,,,,,,,
|
||||
,Staff of Healing (Major),12,1800,1,,,,,,,,,,,,,,
|
||||
,Staff of Healing (True),16,9200,1,,,,,,,,,,,,,,
|
||||
,Staff of Illumination,8,425,1,,,,,,,,,,,,,,
|
||||
,Staff of Illusion,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Illusion (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Illusion (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Necromancy,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Necromancy (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Necromancy (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Staff of Power,16,10000,1,,,,,,,,,,,,,,
|
||||
,Staff of the Magi,20,90000,1,,,,,,,,,,,,,,
|
||||
,Staff of Transmutation,6,230,1,,,,,,,,,,,,,,
|
||||
,Staff of Transmutation (Greater),10,900,1,,,,,,,,,,,,,,
|
||||
,Staff of Transmutation (Major),14,4000,1,,,,,,,,,,,,,,
|
||||
,Verdant Staff,6,225,1,,,,,,,,,,,,,,
|
||||
,Verdant Staff (Greater),12,1750,1,,,,,,,,,,,,,,
|
||||
|
Reference in new issue
Block a user