dashwood.net -

Ryan Stefan's Micro Blog

Multithreading and Run Once Decorator

Jun 112019

I've been coding again and just remembered how well this website works for keeping track of cool tricks I learn. Sometimes it's really hard to find simple and generic examples of things to help teach the fundamentals. I needed to write to a file without opening the text document 1000 times and I finally found a really clean example that helped me understand the pieces.

Edit** Threadpool is a lot easier and you can thread inside a loop:

from multiprocessing.pool import ThreadPool as Pool

threads = 100

p = Pool(threads)
p.map(function, list)

More complicated version:

import threading
 
lock = threading.Lock()
  
def thread_test(num):
    phrase = "I am number " + str(num)
    with lock:
        print phrase
        f.write(phrase + "\n")
 
threads = []
f = open("text.txt", 'w')
for i in range (100):
    t = threading.Thread(target = thread_test, args = (i,))
    threads.append(t)
    t.start()
  
while threading.activeCount() > 1:
    pass
else:
    f.close()

Close something on Scrapy spider close without using a pipeline:

from scrapy import signals
from scrapy.xlib.pydispatch import dispatcher

class MySpider(CrawlSpider):
    def __init__(self):
        dispatcher.connect(self.spider_closed, signals.spider_closed)

    def spider_closed(self, spider):
      # second param is instance of spder about to be closed.

Instead of using an if time or if count to activate something I found a decorator that will make sure the function on runs once:

def run_once(f):
    def wrapper(*args, **kwargs):
        if not wrapper.has_run:
            wrapper.has_run = True
            return f(*args, **kwargs)
    wrapper.has_run = False
    return wrapper


@run_once
def my_function(foo, bar):
    return foo+bar

You can also resize the terminal inside the code:

import sys
sys.stdout.write("\x1b[8;{rows};{cols}t".format(rows=46, cols=54))

I got stuck for a while trying to get my repository to let me login without creating an ssh key (super annoying imo) and I figured out that I added the ssh url for the origin url and needed to reset it to the http:

change origin url
git remote set-url origin <url-with-your-username>

Combine mp3 files with linux:

ls *.mp3
sudo apt-get install mp3wrap
mp3wrap output.mp3 *.mp3

Regex is always better than splitting a bunch of times and making the code messy. Plus it's a lot easier to pick up the code later on and figure out what's going on. So I decided to take my regex to the next level and start labeling groups (I'm even going to give it it's very own tag :3:

pat = r'(?<=\,\"searchResults\"\:\{)(?<list_results>.*)(?=\,\"resultsHash\"\:)'

m = re.match(pat, url)
if m:
    self.domain = m.group('list_results')

Comments