logo
Tags down

shadow

How to read a tarfile object directly with OpenCV


By : user2175236
Date : October 15 2020, 08:10 PM
will be helpful for those in need I think you get a "bytes object" if you do a read() on the result of tar.extract file(). Then you can convert that to a Numpy ndarray and use OpenCV imdecode() to unpack the TIF format from the memory buffer into an image.
This code works for me - though I seem to alternate between reading 266 byte useless files and TIF images from my tarfile - I don't know why that is but if you don't get them, you can just remove the if statement that checks for useless 266 byte files!
code :
#!/usr/bin/env python3

import cv2
import sys
import tarfile                                                                              
import numpy as np

tar = tarfile.open('images.tar','r|') 
for f in tar:
   # Extract an image
   c = tar.extractfile(f).read()
   if sys.getsizeof(c) > 266:
      print(sys.getsizeof(c))
      na = np.frombuffer(c, dtype=np.uint8)
      im = cv2.imdecode(na, cv2.IMREAD_COLOR)
      cv2.imshow('Extracted', im)


Share : facebook icon twitter icon

python write string directly to tarfile


By : Michael
Date : March 29 2020, 07:55 AM
I hope this helps you . I would say it's possible, by playing with TarInfo e TarFile.addfile passing a StringIO as a fileobject.
Very rough, but works
code :
import tarfile
import StringIO

tar = tarfile.TarFile("test.tar","w")

string = StringIO.StringIO()
string.write("hello")
string.seek(0)
info = tarfile.TarInfo(name="foo")
info.size=len(string.buf)
tar.addfile(tarinfo=info, fileobj=string)

tar.close()

Dumping JSON directly into a tarfile


By : Ryan
Date : March 29 2020, 07:55 AM
seems to work fine Why do you think you can write a tarfile to a StringIO? That doesn't work like you think it does.
This approach doesn't error, but it's not actually how you create a tarfile in memory from in-memory objects.
code :
from json import dumps                                                               
from io import BytesIO                                                     
import tarfile                                                                       

data = [{'foo': 'bar'},                                                              
        {'cheese': None},                                                            
        ]                                                                            

filename = 'fnord'                                                                   
with BytesIO() as out_stream, tarfile.open(filename, 'w|gz', out_stream) as tar_file:
    for packet in data:                                                              
        out_stream.write(dumps(packet).encode())                                     

How do I read a tarfile from a generator?


By : Maqsudjon Usmonov
Date : March 29 2020, 07:55 AM
fixed the issue. Will look into that further I had to wrap the generator in a file-like object built on top of the io module.
code :
def generator_to_stream(generator, buffer_size=io.DEFAULT_BUFFER_SIZE):
    class GeneratorStream(io.RawIOBase):
        def __init__(self):
            self.leftover = None

        def readable(self):
            return True

        def readinto(self, b):
            try:
                l = len(b)  # : We're supposed to return at most this much
                chunk = self.leftover or next(generator)
                output, self.leftover = chunk[:l], chunk[l:]
                b[:len(output)] = output
                return len(output)
            except StopIteration:
                return 0  # : Indicate EOF
    return io.BufferedReader(GeneratorStream())
stream = generator_to_stream(any_stream)
tar_file = tarfile.open(fileobj=stream, mode='r|*')
#: Do whatever you want with the tar_file now

for member in tar_file:
    member_file = tar_file.extractfile(member)

python tarfile unpredictable error tarfile.ReadError: empty header


By : Anria B.
Date : March 29 2020, 07:55 AM
may help you . This exception is raised when the buffer length is zero while parsing headers for the tarfile. It is raised for an empty archive.
Reference: https://github.com/python/cpython/blob/master/Lib/tarfile.py#L1028 http://bugs.python.org/issue6123

How can I read an image from Azure blob storage directly using Opencv without downloading it to a local file?


By : Mạnh Nguyễn Hữu
Date : March 29 2020, 07:55 AM
will be helpful for those in need Per my experience, you can try to use get_blob_to_bytes method to download the blob as a byte array and covert it to a opencv image, as my sample code below.
code :
from azure.storage.blob import BlockBlobService

account_name = '<your-storage-account>'
account_key = '<your accout key>'
block_blob_service = BlockBlobService(account_name, account_key)

container_name = 'mycontainer'
blob_name = 'test.jpg'
blob = block_blob_service.get_blob_to_bytes(container_name, blob_name)

import numpy as np
import cv2
# use numpy to construct an array from the bytes
x = np.fromstring(blob.content, dtype='uint8')

# decode the array into an image
img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)
print img.shape

# show it
cv2.imshow("Image Window", img)
cv2.waitKey(0)
Related Posts Related Posts :
  • I'm getting positional argument in Django rest framework APIView class empty. Why? And how to pass value into it?
  • Create an array according to index in another array in Python
  • Matplotlib multiple Y-axes, xlabels disappear?
  • feedparser for reddit returning empty
  • physical dimensions and array dimensions
  • can't get my program to return to main loop
  • how to read image into tensor from url directly
  • Can't find a combination of keywords on an xml page using python and beautiful soup
  • Find the rotation of a quad (4 points, planar)
  • Class method input variables
  • Pandas Dataframe, how to group columns together in Python
  • What does "auth.User" in Django do?
  • Python - Get Last Element after str.split()
  • How to access a variable in one python function in another function
  • Manually computed validation loss different from reported val_loss when using regularization
  • Filtering with a only one conditional
  • How to set specific faker random string of specific length and using underscores for spaces?
  • seaborn FacetGrid+map_dataframe fails (but not when using map)
  • How to get GraphQL schema with Python?
  • Python - How to send values between functions once
  • Loop sum find and multiple
  • Map & append multiple values (per each key) from a dict to different columns of a dataframe
  • Python list of dictionaries incrementation error
  • Filtering Spark Dataframe
  • pytest: How to test project-dependent directory creation?
  • Python Group by and Sum with a Blank space
  • Reorder and return the whole of nested dictionary
  • Finding element from one list in nested second list
  • Calculating AUC for Unsupervised LOF in sklearn
  • Storing Specific Whole Numbers - Python
  • Simulate SHL and SHR ASM instructions in Python
  • AttributeError: type object 'DirectView' has no attribute 'as_view'
  • Iterate through list and print 'true' if list element is of a certain type
  • Web scraping: How to test if the root tag has a particular CSS class?
  • Bar Graph in Python with 2 scales (y-axis) with matplotlib import
  • Use of Breakpoint Method
  • pdf file generated using reportlab , getting saved as plain text
  • Pandas DataFrame: programmatic rows split of a dataframe on multiple columns conditions
  • Combine the data based on common index with dates
  • Accessing Tags within HTML comments
  • how to conditionally change array values
  • Numpy failing in Python3 Google App Flexible Engine
  • Fastest way to delete/extract a submatrix from a numpy matrix
  • Pandas dividing distance by timedelta
  • Enforce minimum spacing between non-zero elements in array
  • Convert web scraped string list to formatted CSV
  • How to clear value from cell if value in another column is present?
  • what use tensorflow estimator create multi-input
  • Selenium: Find an element stored in a container
  • specify *.pyd output path?
  • Looping through pixels with Cython still slow
  • Python - Recursive Function does not work with len() despite the returned value being a list
  • Having trouble reading Pandas dataframe with SciLearn Kit
  • Reshape vertical series to horizontal in Python
  • Removing background from galaxy photos using Python
  • Failed to verify: !m_var2expr.empty() on z3 python
  • what does [:, :, ::-1] mean in python?
  • Conditional loop for pandas dataframe rows
  • Check the value in next row in order to apply certain rules to current rows in Python
  • Python getting the grouped values from positive look ahead
  • shadow
    Privacy Policy - Terms - Contact Us © voile276.org